zq: GESIS Panel.dbd Core Study Module – Monitoring quality: survey experience & mode characteristics

Study Code

zq

Version and date of last revision

Version 1; October 16, 2025

Study Title

GESIS Panel.dbd Core Study Module – Monitoring quality: survey experience & mode characteristics

Subject Classification

Survey methodology

Keywords

Survey experience, survey mode, duration, interview situation

Data Collection Waves

rec, ka

Principal Investigators

Name(s) and affiliations of principal investigators at time of submission Persistent digital identifier(s) of principal investigator(s)
Bella Struminskaya, Utrecht University http://scholar.google.de/citations?user=UBs7GvsAAAAJ
Michael Bosnjak, University of Trier https://orcid.org/0000-0002-1431-8461
Lars Kaczmirek, University of Vienna https://www.researchgate.net/profile/Lars-Kaczmirek

Abstract

This study is a part of the GESIS Panel.dbd Longitudinal Core Study series. One aim of the GESIS Longitudinal Core Studies is to measure frequently demanded characteristics of enduring interest to the social sciences. Another aim is to assess indicators and proxy variables of data quality, facilitating estimation of potential survey errors and biases. As opposed to other GESIS Panel.dbd Longitudinal Core Study modules which are fielded once per year, this study is fielded in every questionnaire.

This study module belongs to the second category and focuses on quality indicators. It has three distinct areas of focus: (1) respondent’s assessment of the questionnaire; (2) characteristics of the survey situation, and (3) respondent’s feedback.

The respondent’s assessment of the questionnaire has several measures: six about the questionnaire (interesting, diverse, important for science, long, difficult, too personal), and three about whether the questions were difficult to understand, whether it was difficult to find an answer, and the overall evaluation. The questions were developed and tested in the project GESIS Online Panel Pilot (Kaczmirek, Bandilla, Schaurer & Struminskaya, 2014).

The first goal of the evaluation matrix was to measure how respondents perceive the questionnaire to understand how well respondents differentiate between the questionnaires. The second goal was to study the effects of respondents’ survey experience on panel attrition (see Struminskaya, Kaczmirek, De Leeuw, in preparation). The adjectives which make up the items in the evaluation matrix were developed on the basis of the content analysis of the data of over 4000 respondents in six other unrelated surveys where respondents completed a sentence similar to „I felt/perceived the questions were…” (Kaczmirek, Baier, & Zuell, 2010). The item „important for science” was not based on the content analysis but was included in addition to capture the influence of the sponsor because previous research showed that academic and government sponsorship („science”) opposed to commercial sponsorship („company research”) increases respondent cooperation (e.g., Walston, Lissitz, & Rudner, 2006). In the GESIS Panel, the scale for the matrix question was changed from an original 4-point to a 5-point scale with the inclusion of the middle category.

Another set of survey evaluation items developed by Jon Krosnick is included to cross-validate and supplement the GESIS Panel survey evaluation items. The additional set of items focuses on respondents’ effort and subjective experience of filling out the questionnaire.

The items describing the survey situation were developed ad hoc and include the information on presence of other persons during the response process, the place of participation (home or other place), and for the online mode the question on the device the respondent completed the survey with (PC, laptop, mobile phone etc.).

The item „remark” is a replication from GESIS Panel Online Pilot, where it was tested, and allows for gathering respondents’ feedback on the dimensions not covered by the survey evaluation items.

Measured constructs/concepts and corresponding data collection waves

Constructs/concepts Corresponding indicators (survey measures) Data Collection Waves
Questionnaire Assessment Q: Der Fragebogen war abwechlungsreich.
R:
  • Überhaupt nicht
  • Eher nicht
  • Teils, teils
  • Eher
  • Sehr
Q: The questionnaire was varied.
R:
  • Not at all
  • Rather not
  • Partly, partly
  • Rather
  • Very
Q: Der Fragebogen war interessant.
R:
  • Überhaupt nicht
  • Eher nicht
  • Teils, teils
  • Eher
  • Sehr
Q: The questionnaire was interesting.
R:
  • Not at all
  • Rather not
  • Partly, partly
  • Rather
  • Very
Q: Der Fragebogen war lang.
R:
  • Überhaupt nicht
  • Eher nicht
  • Teils, teils
  • Eher
  • Sehr
Q: The questionnaire was long.
R:
  • Not at all
  • Rather not
  • Partly, partly
  • Rather
  • Very
Q: Der Fragebogen war schwierig.
R:
  • Überhaupt nicht
  • Eher nicht
  • Teils, teils
  • Eher
  • Sehr
Q: The questionnaire was difficult.
R:
  • Not at all
  • Rather not
  • Partly, partly
  • Rather
  • Very
Q: Der Fragebogen war wichtig für die Wissenschaft.
R:
  • Überhaupt nicht
  • Eher nicht
  • Teils, teils
  • Eher
  • Sehr
Q: The questionnaire was important for science.
R:
  • Not at all
  • Rather not
  • Partly, partly
  • Rather
  • Very
Q: Der Fragebogen war zu persönlich.
R:
  • Überhaupt nicht
  • Eher nicht
  • Teils, teils
  • Eher
  • Sehr
Q: The questionnaire was too personal.
R:
  • Not at all
  • Rather not
  • Partly, partly
  • Rather
  • Very
Q: Wie hat Ihnen die Befragung insgesamt gefallen?
R:
  • Überhaupt nicht
  • Nicht so gut
  • Mittelmäßig
  • Gut
  • Sehr gut
Q: How did you like the survey overall?
R:
  • Not at all
  • Not very good
  • Mediocre
  • Good
  • Very good
Q: Wie schwierig war es für Sie, auf die Fragen in diesem Fragebogen eine Antwort zu finden?
R:
  • Äußerst schwierig
  • Sehr schwierig
  • Mäßig schwierig
  • Etwas schwierig
  • Überhaupt nicht schwierig
Q: How difficult was it for you to find an answer to the questions in this questionnaire?
R:
  • Extremely difficult
  • Very difficult
  • Moderately difficult
  • Somewhat difficult
  • Not difficult at all
Q: Wie schwierig war es für Sie, die Fragen in diesem Fragebogen zu verstehen?
R:
  • Äußerst schwierig
  • Sehr schwierig
  • Mäßig schwierig
  • Etwas schwierig
  • Überhaupt nicht schwierig
Q: How difficult was it for you to understand the questions in this questionnaire?
R:
  • Extremely difficult
  • Very difficult
  • Moderately difficult
  • Somewhat difficult
  • Not difficult at all
 		rec, ka
Survey Experience Q: Haben Sie in den letzten 12 Monaten noch an anderen Umfragen teilgenommen?
R:
  • Nein
  • Ja, und der Name der Umfrage war ...
Q: Have you taken part in any other surveys in the last 12 months?
R:
  • No
  • Yes, and the name of the survey was ...
Q: Neben unserem GESIS Puls gibt es andere Umfragen, an denen man als Mitglied regelmäßig teilnehmen kann. Solche Umfragen, die eine Mitgliedschaft voraussetzen, werden Panels genannt.Sind Sie neben der Mitgliedschaft in unserem GESIS Puls auch Mitglied in einem anderen Panel?
R:
  • Nein, und ich kenne auch keine anderen Panels.
  • Nein, ich kenne aber andere Panels.
  • Ja, und zwar ...
Q: In addition to our GESIS Puls, there are other surveys in which you can regularly participate as a member. These surveys, which require membership, are called panels. Are you a member of any other panel in addition to our GESIS Puls?
R:
  • No, and I don't know any other panels.
  • No, but I know other panels.
  • Yes, namely ...
 		rec
Respondents' feedback Q: Haben Sie noch weitere Anmerkungen?
R:
  • keine Angabe gemacht
  • Angabe gemacht
Q: Do you have any further comments?
R:
  • no response given
  • response given
 		ka
Survey Situation Q: Mit welchem Gerät haben Sie die Fragen beantwortet?
R:
  • PC bzw. Laptop
  • Tablet-PC
  • Smartphone
  • Anderes Gerät, und zwar: ...
  • Anderes Gerät, und zwar: ...
Q: Which device did you use to answer the questions?
R:
  • PC or laptop
  • Tablet PC
  • Smartphone
  • Other device, namely: ...
  • Other device, namely: ...
Q: Von wo aus haben Sie an dieser Befragung teilgenommen?
R:
  • Von Zuhause
  • An einem anderen Ort
Q: Where did you take part in this survey?
R:
  • From home
  • In another place
Q: Waren Sie bei der Beantwortung der Fragen allein oder waren weitere Personen anwesend?
R:
  • Ich war allein.
  • Andere Personen waren anwesend.
Q: Were you alone when answering the questions or were there other people present?
R:
  • I was alone.
  • Other people were present.
 		rec, ka

References

Kaczmirek, L., Baier, C., & Züll, C. (2010). Wie empfinden Teilnehmer die Fragen in Online-Befragungen? Entwicklung eines Diktionärs für die automatische Codierung freier Antworten. In M. Welker & C. Wünsch (Eds.), Die Online-Inhaltsanalyse: Forschungsobjekt Internet (Vol. 8, pp. 191–223). Halem.
Kaczmirek, L., Bandilla, W., Schaurer, I., & Struminskaya, B. (2014). GESIS Online Panel Pilot: Multitopic introductory wave (survey 1)GESIS Online Panel Pilot: Einführende Mehrthemen-Welle (Umfrage 1). GESIS Data Archive. https://doi.org/10.4232/1.11570
Walston, J. T., Lissitz, R. W., & Rudner, L. M. (2006). The influence of web-based questionnaire presentation variations on survey cooperation and perceptions of survey quality. Journal of Official Statistics, 22, 271–291. https://api.semanticscholar.org/CorpusID:125006432