There are many types of measurement approaches appropriate for research on service and learning.
Surveys are one of the most commonly used research tools at CSL and in general. They may be conducted in various ways—in person, by telephone, by email, through the Internet, or on paper. CSL most often uses self-report measures in its surveys. Students report on their own attitudes, opinions, behaviors, feelings, or beliefs. Self-report measures are very useful for many research purposes because they obtain information directly from the respondent. However, there are some weaknesses to using self-report measures. For instance, responses are subjective and may not coincide with ratings given by other sources of information. Additionally, responses may be subject to social desirability bias (the tendency for a person to give responses that are normative and present oneself in a positive manner).
Interviews are another research tool that is especially useful at the exploration stage, or for qualitative research. Conducted either in person or by telephone, interviews are similar to surveys, but often are used to assess information in more depth than is possible in a survey. Questions in an interview can be open ended or structured (predetermined content and order). Likewise, the responses can be open ended or close ended (respondent must choose among the categories provided by the interviewer).
Focus groups are interviews conducted in a group setting. An advantage of focus groups is that participants can interact with each other. Time savings is another advantage since the interview is conducted in a group instead of one-on-one. A potential weakness of focus groups is that the format may suppress information from some respondents. Another shortcoming of focus groups is that some participants may not have time or feel free to make completely honest comments in front of others. Finally, focus groups may not be as useful as interviews for getting in-depth information about an individual’s experiences.
Document review of course syllabi, faculty journals, meeting minutes, strategic plans, annual reports, mission statements, or reflection products may be required in a research project. These documents are a substantial source of information about programs and organizations.
Reflection products are commonly used for service and learning assessment. Case studies, journals, portfolios, papers, presentations, and interviews are all forms of reflection. Campus Compact provides very useful information on how to structure reflection to enhance student learning. Reflections are usually analyzed through content analysis or rubrics.
Rubrics are scoring tools for subjective assessment, enabling more standardized evaluation of reflection products or other artifacts. Generally occurring in the form of a matrix, rubrics have the following characteristics:
- Traits or dimensions that serve as the basis for judging products
- Definitions or examples to illustrate the traits or dimensions
- A scale of values on which to rate the traits
- Standards or examples for each performance level
CSL has made much use of rubrics in its research and assessment, developing the following: