Guidelines for effective survey development, administration, and analysis

Interim guidance - More detailed information is forthcoming.
Last updated October 2024

Feedback from our students, faculty, and staff is a gift. It is our responsibility to conduct meaningful and well-designed surveys so that we respect respondents’ time and use their feedback intentionally. At the same time, conducting a survey is time-intensive and resource-intensive, sometimes requiring weeks or months of thoughtful planning.

This document aims to help university employees who are conducting surveys as part of their work design and administer surveys effectively and analyze results intentionally. It contains interim guidance on survey development, administration, and analysis at UMD. This guidance was developed by a group of experts from the Division of Academic Affairs (Office of the University Registrar and the Office of Institutional Research, Planning & Assessment), Division of Administrative Affairs, Division of Information Technology (Academic Technology and Innovation), and the Division of Student Affairs.


Who is this guidance for?

This guidance is geared toward university employees who are seeking student, staff, and/or faculty feedback via surveys that meet any of the following criteria:

  • Generalize results to the broader UMD population or a subpopulation
  • Target specific subpopulations, such as by enrollment status, gender, race/ethnicity, first-generation status, etc.
  • Use results for institutional measures and improvement
  • Share results with the public (e.g., posting on the UMD website)
  • Customer service and point-of-service surveys

This guidance is not geared towards (although may still benefit) employees who are conducting quick “pulse” surveys or evaluations during a scheduled program; or students who are conducting surveys as part of their coursework. This guidance does not replace review by the Institutional Review Board (IRB) for projects that require IRB approval.


Survey guidance

We outline 6 steps to conducting an effective survey below. Click on the hyperlinks to go directly to each step.

  1. Define your purpose
  2. Decide who you will survey
  3. Create a plan for confidentiality/anonymity, data security, and IRB approval (if necessary)
  4. Design your survey
  5. Administer your survey and collect data
  6. Analyze, interpret, and share your results

Consistent with the guiding principles of UMD’s Strategic Plan, attending to diversity, equity, and inclusion needs to be embedded in all of our practices. Therefore, equity-centered assessment practices need to be incorporated at all stages of your process from design, implementation, interpretation, and action planning.

Additional campus resources are listed at the end of this document.

Step 1. Define your purpose

  • What is the goal of your survey? What do you want to learn? If possible, make a list of specific questions you are trying to answer and decisions the survey will help inform.
  • Consider whether the information you are trying to collect is available elsewhere (e.g., reports.umd.edu, data from information systems). Self-reported survey data about behavior is usually not the most accurate way to gather data about behavior and needs to be interpreted with caution.
  • Are there alternatives to a survey that would get you better information (e.g., interviews, focus groups, etc.)?
  • To determine if a survey is the most effective way to collect information, consider:
    • How will you use this information you collect?
    • Does it provide useful information for academic planning purposes and/or for those providing services to students?
    • Does it provide useful information on the experience of students, staff, and/or faculty that we cannot get elsewhere?
    • Could the results negatively impact the university?
    • Would a survey divert important university resources away from other projects?
    • Is your goal to learn about opinions and perceptions? A survey can be an effective tool for gathering these data. On the other hand, a survey is usually not the most accurate way to gather data about behaviors or actions.
  • Design an analysis plan for the survey results in advance. What demographic variables (e.g., college of major, class standing, gender) would be meaningful and actionable to analyze? Creating a rigorous plan for analysis up-front will help avoid “fishing” for results once you get the survey results.
  • Consider triangulating survey data with other data sources you have access to.

Step 2. Decide who you will survey

  • This may differ depending on the types of actions you intend to take based on the results.
  • Rarely does a survey require distribution to a full population to get valid results. You can use sampling techniques, such as random sampling, and then generalize to the population.
  • If you are currently conducting a population survey with a low response rate, consider reevaluating your process.
  • To request a list of emails, complete the Office of the Registrar Data Request Form (for student contact information) and UHR Data Services (for employee contact information).

Step 3. Create a plan for confidentiality/anonymity, data security, and IRB approval (if necessary)

  • Student survey data must be kept confidential. Data shared must not include student identifiers such as social security number, UID, or name; reports (paper, electronic, or verbal) using the data must not identify individual students. UMD’s Institutional Review Board (IRB) has additional information on maintaining confidentiality.
  • An anonymous survey would require that the survey has no way to link a respondent with their responses. This may be more appropriate if the survey topic is sensitive.
  • Summary data with small cell sizes should not be reported if it could potentially reveal information about an individual student.
  • Decide whether you need IRB approval for your survey. How do you plan to use the results? Who do you plan to share them with? Is your survey part of a research project?

Step 4. Design your survey

Step 5. Administer your survey and collect data

  • Timing: Avoid administering a survey during finals, midterms, or the first and last weeks of the semester. This timing can impact your response rate.
  • Communicating about your survey: Make survey invitations and reminders short and personal (e.g., include the recipient’s name if the survey isn’t anonymous; have the survey be sent from someone whose name recipients will recognize).
  • Incentives: Consider when and what incentives might be appropriate. Depending on how the incentive is structured, it might change someone’s responses or how they complete the survey (e.g., an incentive that rewards the first 100 people who complete a survey might motivate survey-takers to take the survey very quickly, rather than thinking through each question).

Step 6. Analyze, interpret, and share your results

  • Your analysis should relate to your goals (see step 1). For each goal, state what you found using data.
  • Review data before you start analyzing it to ensure that values are encoded properly, there aren’t any unexpected responses, etc.
  • Oftentimes, the simplest analysis and statistics will answer your question. It doesn’t need to be complex!
  • When sharing your results, consider the audience. You may want to provide different reports, presentations, etc., for different audiences. Which audiences care about your methodology? Which audiences want to know about your process? How much background knowledge does your audience have?