Which questions should be asked in questionnaires?

3.5 Structure of the questions

3.5.1 Functions of questions

According to Kleber (1992, p.217), questions can be classified according to their function:

  • Contact and introductory questions to get started with the survey,
  • Transitional and preparatory questions for changing topics,
  • Distraction and buffer issues to reduce spill-over effects,
  • Filter questions to skip over possibly irrelevant questions,
  • Maneuvering and concentration issues to loosen up long presentations,
  • Motivation questions to strengthen self-confidence and reduce inhibitions,
  • Control questions as checking the truth of the answers or making contradictions visible.

These classified questions are used to structure the content-relevant questions that make up the actual survey.

3.5.2 Relation of question type and expected information

The focus here is on what information should be collected through the questions. The distinction is important for the choice of the answer scale (see Chapter 3.6.2). According to Schnell et al. (1999, p.303 ff), a distinction can be made between four types of questions:

  • Attitude or opinions,
  • Beliefs,
  • Behavior,
  • Properties.

Questions about attitudes or opinions

This type of question relates to “desirability or the negative or positive assessment that respondents associate with certain statements” (Schnell et al. 1999, p. 303). This can be expressed both in the question itself and in the answer specifications.

Examples of attitude questions: Should the university offer more creative thinking courses? o desirable o undesirable Should the monarchy be reintroduced? o yes o no The commercials on television are too long. o right o wrong

Asking about beliefs

The assessment that is required in this question is based solely on the conviction of the respondent. So there are questions or problem areas where right or wrong does not matter, but how the respondent perceives reality.

Examples of questions of conviction: Commercials on private television now take up at least 1/3 of the airtime. o correct o incorrect How much built-up area do the university facilities in Tübingen take up? o 1/10 o 1/5 o 1/3 o ...

Questions about behavior

In questions of behavior, reference is always made directly to the behavior of the questioner and not to any general behavior of groups. This is how they differ from questions of conviction. Examples of behavioral questions:

Would you accept more advertising on public television if the GEZ fee was no longer applicable? o yes o no Have you ever taken part in a course on creative self-realization? o never o once o a few times o often o regularly

When it comes to questions of behavior, it should always be borne in mind that the behavior is described and that it does not always have to correspond to reality. On the one hand, the self-perception of the participants partly does not correspond to the observable real behavior (Eckert 1999, p. 379), on the other hand, wishful thinking of the interviewee can also be the basis. This is particularly important when it comes to future-oriented behavioral issues.

Questions about properties

The characteristics of the person are asked for, usually personal and demographic data. These are queried in most questionnaires, partly to determine relationships between demographic characteristics of people or groups and their attitudes, convictions and behavioral patterns.

Examples of property questions: How much money do you have per month? o 0-499 o 500-999 o 1000-1900 o 2000 and more How old are you? o under 18 o 18-29 o 30-45 o 46-60 o over 60

Percentage distribution of the types of questions mentioned

According to Schnell et al. (1999, p.306) the following distribution is estimated for a "normal" standardized survey:

33% attitude questions,
20% behavioral questions,
47% respondent characteristics
(Demographics).


Fig. 12: Percentage distribution of the types of questions

"In general, one can assume: the more unprofessional the investigation, the higher the proportion of questions about attitudes." (Schnell et al. 1999, p.306) A reason for the statement is given by Schnell et al. however not mentioned.

3.5.3 Open, half-open and closed questions

A broad distinction can be made between two structural types of questions - open and closed questions. The mixed form, called the hybrid question, should also be mentioned. After considering the two structure types, their use in practice is particularly noteworthy.

Closed questions

The answer categories are given for closed questions. The respondent therefore moves within a defined grid.

shape example
Yes - no form Iron has a lower specific weight than copper.
o yes o no
Correct answer form Underline names of deciduous trees: spruce, oak, larch, beech, poplar.
Best-answer form Munich is located in Europe, Germany, Bavaria, southern Germany.
Allocation form Which tools are mainly required in which professions? Brush (a) (b) (c) (d) (e) Knife (a) (b) (c) (d) (e) Plane (a) (b) (c) (d) (e) (a ) Dentist; (b) bricklayer; (c) butcher; (d) carpenter; (e) painter
Rating scales of subjective assessment How did you feel about the pace of the course? too fast | ------ | ------ | ------ | ------ | too slow

Table 9: Overview of various forms of closed questions

According to Schnell et al. (1999, p.173 ff.) There are different forms of building up the scaling. Scales after Thurstone, Likert, Guttman, Rasch and Magnitude are named. Since the 5- or 7-point Likert scale is the most frequently used scaling method in the social sciences, only these are discussed below. When it is spoken of, however, more than just a certain number of classifications is meant. There is also a certain method of finding items. The term item refers to the question about the scale itself. The Likert scale is outlined below, as it is relevant for questioning.

advantages disadvantage
+ easy evaluation
  • Danger of guessing instead of real answering
  • labor intensive formulation
  • great preparation effort
  • answers with 'yes / no' or 'right / wrong' can memorize the inaccurate assertions
  • Choices for the respondent may be irrelevant

Table 10: Advantages and disadvantages of closed questions

Excursus: the process of finding items on the Likert scale

The process begins with the collection of a large number of items. In the literature, 100 is mentioned as the number. These items are the same in terms of content, only the verbal descriptions vary.

As an example according to Schnell et al. (1999, p.184):

"1. Somehow, I think, my work is important 2. At work I often feel somehow empty 3. I have the feeling that I am doing something useful in my work ... 100. I learn something new every day in my work "

Both positively and negatively formulated questions are represented, the "signs" of which must later be taken into account in the statistical evaluation. These questions are presented to test subjects. The result of the item finding is all the better, the greater the number of test subjects. A sample of 2505 people was available in the study mentioned. From their point of view, the test subjects now rate the items on a 5- (or 7-) level scale. The scale used can be structured as follows: applicable, more or less applicable, neither / nor, rather inapplicable, inapplicable. The best items are filtered out using statistical methods, which will not be described in more detail here. The reliability is determined with the Cronbach Alpha.

The process of determining the most suitable items is complex and time-consuming. For this reason, the process is avoided by many questionnaire creators, and therefore the items often arise from "the unrestrained imagination of the scale constructors" (Schnell et al. 1999, p.173).

A few rules of thumb have been developed for the formulation of the items. This is discussed in more detail in the further course of the chapter under point 3.5.4.

In personal conversations with several psychologists, the unanimous opinion emerged that 5 levels are normally the standard when using Likert scales, since the gain in knowledge is negligible with 7-level scales. Most people cannot differentiate their answers that finely.

Half-open questions

In contrast to closed questions, there is no specification of the answer categories for half-open questions, but the respondent has to write out his answer. To distinguish it from open questions, it could be argued that the answer usually consists of one or a few words.

shape example
Question form Which river is Tübingen on?
Supplementary form Water consists of hydrogen and ...
Error correction form Yesterday I write a letter.

Table 11: Different forms of half-open questions

advantages disadvantage
+ easy formulation - more difficult evaluation
+ little preparation effort

Table 12: Advantages and disadvantages of half-open questions

Open questions

In the case of open questions, the respondent can freely formulate his or her answer, attitude or conviction and is not forced into a grid by given answer options. This type of question trusts the respondent to have a differentiated self-perception, ability to express himself, motivation and honesty (Gerl 1983, p.65).

shape example
Question form What do you think could be done to improve the quality of the data center's courses?

Table 13: Form of the open questions

The open form brings particular advantages in the development of the questionnaire: After the pretest, the answers obtained can be evaluated and open questions can be transformed into closed questions. The answer variants and ideas of a wide variety of people who have gone through the pretest are taken into account, and the closed questions do not arise solely from the limited imagination of an individual.

advantages disadvantage
  • Answers are given within the own reference system (Schnell et al. 1999, p.309)
  • no guidance through given answer options
  • new aspects, which were not taken into account in the development of the questionnaire, are pointed out by the “person concerned”
  • Question form can be used ideally for development
  • Differences in the verbal / written articulation skills of the respondents
  • significantly higher evaluation effort
  • difficult to evaluate for inexperienced users (Beywl / Schepp-Winter 2000, p. 51)
  • the answers are difficult to compare (ibid.)

Table 14: Advantages and disadvantages of open questions

Considerations on the use of open and closed questions

The disadvantages of open questions have become increasingly problematic for use in practice over the years. In 1983 (see Schnell et al. 1999, p.310), for example, when analyzing a large number of studies, Caplovitz shows that the proportion of open questions has decreased drastically from 16% to 3% since the 1940s.

My assumption is that due to the high effort involved in the evaluation and the impossibility of their automation, open questions will decrease even further. Above all, the costs, which are associated with a high expenditure of time, should contribute to this.

The open questions for the development and pretests of questionnaires will continue to be of interest, as they are the only way to broaden the horizon of the questionnaire developer to the target group.

3.5.4 Formulation of the questions

For the formulation of the questions, some rules have emerged, which Edwards (in Mummendey 1995, p.63) summarizes as follows:

Statements should be avoided

  • that relate to the past rather than the present,
  • describe the facts or can be understood as a description of the facts,
  • which cannot be clearly interpreted by the respondent,
  • that do not relate to the attitude that is at stake,
  • all or none of the respondents agree.

Statements should

  • cover the entire affective area of ​​the attitude,
  • be simple, clear and direct,
  • be short and rarely exceed 20 words,
  • always contain only one object,
  • do not contain words like "all", "always", "nobody" and "never",
  • Words like "only", "even" and "hardly" contain only in exceptional cases,
  • consist of simple sentences and not of compound sentences or compound sentences,
  • do not contain any words that could be incomprehensible to the person asked,
  • do not contain double negations.

Kleber also made the requirement that the question and the formulation of the answer must be adapted to the target group concerned in order to avoid overburdening the people interviewed (cf. Kleber 1992, p.217).

language

Anyone who varies the language in the questionnaire will find "that even linguistic nuances, the smallest changes (...) can acquire great importance" (Mummendey 1995, p. 143). For example the word 'often'. When is something often? The most diverse assessments of people of such a word make the dimensions of semantic gray areas in the language aware.

Positive response tendencies and the problem of reversibility

It has to be stated again and again that parts of the respondents tend to affirmative regardless of the content. In order to get a grip on this problem of such a bias in results, Mummendey looked at the reversibility of the items. The reversal of a statement is associated with various problems. For example, the example given by Mummendey (1995, p. 144) “I tend to take things lightly” has a different semantic quality in the formulation “I tend to take things hard.” Of course, a reversal could also result The insertion of a negation can be effected in different places, e.g. “I do not tend to take things lightly.” However, reversing items turns out to be difficult. Measuring the correlation with the initial scale shows the problem. Mummendey (1995, p.145) states that “it is almost impossible, but at least very time-consuming, to precisely change even apparently simple linguistic formulations in a desired manner” (Mummendey 1995, p.145).

One suggestion is to include both positive and negative formulations in an item. The example given would then look like this: "Do you tend to take things lightly or to take things difficultly?"

No double negations

It is important to avoid double negations. The difficult comprehensibility that often results from this leads to ambiguous items and thus to uncontrolled results. The question “Do you tend not to take things lightly?” Requires the respondent to deal more precisely with the question, which for him actually represents one of many. The first part of the respondent's analytical work is to make the question understandable. What is actually being asked? Do I tend not to take things lightly, i.e. difficultly? So if I have a tendency to take things lightly, the best way to answer the original question is through simplifying the question. This intellectual work distracts the respondent from his actual task - answering the questions quickly and accurately. In part, the double negation is not really understood, but conventionalized as a single negation.

Use neutral terms

The subliminal evaluation of many terms prohibits their use in the questionnaire. For example, the term 'capitalist' has a far more negative connotation than 'capital provider', which can be used in exactly the same way, depending on the question. It is therefore imperative to choose a neutral formulation, as otherwise the answers given can be skewed in a certain direction. The language habits of the target group should be taken into account when constructing the questionnaire. The term capitalist is usually used differently for stockbrokers than for socially marginalized groups.

Structure of the question and answer formulation

Gerl demands that “the first and most important principle” is that each respondent can only “speak for himself”. Therefore, when formulating the questions, it is essential to ensure that personal attitudes are asked instead of assumptions about attitudes of others - that is, only about subject-related statements.

For example: "Are you interested in the topic?" Is better than "Are the participants interested in the topic?" (Cf. Gerl 1983, p.31)

Positive examples from Gerl (1983, p.43): “- In this learning group I feel free and relaxed. - With my opinion, I prefer to keep behind the mountain here. - I enjoyed working with the others.- I would like to do a course again with the same people. "

Further requirements for the structure of question and answer formulations are described below.

Formulation of questions

On its website (Gräf et al. 2001, Internet source), Gräf offers a very dense catalog of requirements for the formulation of questions. The following list is based on this and was structured and summarized by me with key words.

Short question
The questions should be formulated as briefly as possible
No answer options
An answer can already be given by leading questions and the result will not reflect the opinion of the respondents. Therefore, leading questions must be avoided at all costs.
Concrete and clear
Answers to questions that relate to specific situations are usually more reliable than general questions. Therefore, one must formulate as concretely and clearly as possible.
Examples or explanations
Unknown issues should be introduced with an explanation or an example. This is necessary if the respondent might be unfamiliar with the facts.
Describe situations real and completely
When describing situations, the time, place and context should be given. Terms like young, old, rich mean something different for everyone. Therefore, specific questions should be asked, e.g. B. "When you started school ..." instead of "When you were young ...".
Hypothetical formulations
Hypothetical formulations should be avoided, because the result should not only be hypothetical in nature. With a hypothetical formulation, the respondent is more concerned with considering the probability of a hypothetical result occurring, rather than with the answer.
Example: "You find 100 DM ..." instead of "Suppose you find 100 DM ..."
One-dimensionality
Multiple dimensions within a question should be avoided, otherwise the answer may be multidimensional, and during the evaluation there is usually no longer any possibility of finding out what exactly the answer relates to.
Multi-dimensional questions should better be broken down into several questions.
Negative example: "Do you only use the Internet privately or also for business?"
Complicated considerations / calculations
The more complicated the considerations the respondent has to make in order to obtain a correct answer, the more likely it is that the quality of the answer will deteriorate. Example: "What percentage of your free time do you spend talking to your friends?" The question is complex for several reasons. When does free time begin? Do you also have to eat and sleep? What are friends Do you also include close friends? What percentage is that then in the end? Such a question also presupposes that the respondent seriously takes the trouble to 'calculate' the required result.
Positive and negative answer options
The question should include both the positive and negative answer options. This weakens answer requirements.
Example: “Should 16-18 year olds be prohibited from consuming alcohol?” It would be better to formulate it as follows: “Should 16-18 year olds be prohibited or permitted?”
Approval / rejection
According to method research, questions aimed at approving or rejecting a statement are particularly predestined for distorting answer tendencies, caused by sources of error such as approval tendencies. The recommendation is to keep the proportion of such questions low.
Delicate and sensitive topics
The tricky and sensitive topics in our culture include, for example, questions about income, crime and sexuality. Gentle expressions should be used for questions on these topics. For example, the phrase 'not telling the truth' can be used instead of the harsh word 'lying'. It is important that the respondent does not get the feeling that he has to reckon with a loss of prestige for his answers (see chapter 3.4.3).