SOEP-Core - 2016: Report of survey methodology and fieldwork for M3 and M4

Huber, Simon; Glemser, Axel Research Report SOEP-Core - 2016: Report of survey methodology and fieldwork for M3 and M4 SOEP Survey Papers, No. 1049 Provided in Cooperation with: German Institute for Economic Research (DIW Berlin) Suggested Citation: Huber, Simon; Glemser, Axel (2021) : SOEP-Core - 2016: Report of survey methodology and fieldwork for M3 and M4, SOEP Survey Papers, No. 1049, Deutsches Institut für Wirtschaftsforschung (DIW), Berlin This Version is available at: https://hdl.handle.net/10419/242105 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by-sa/4.0/ Simon Huber, Axel Glemser (Kantar Public) Series B - Survey Reports (Methodenberichte) SOEP-Core – 2016: Report of Survey Methodology and Fieldwork for M3 and M4 1049 2021 Simon Huber, Axel Glemser (Kantar Public) Munich, 2021 To DIW Berlin Mohrenstraße 58 10177 Berlin Submitted by: Simon Huber Kantar GmbH Kantar Public Deutschland Landsberger Str. 284 80687 München Your contact persons: Simon Huber Research Director Tel: 089 5600 - 1414 E-Mail: [email protected] Axel Glemser Senior Director Tel: 089 5600 - 1486 E-Mail: [email protected] Munich, 28 June 2021 315110938, 315112121 SOEP 2016 Wave Report+ Samples M3/4 © Kantar 2021 2 Contents 1 Introduction 3 2 Background Samples M3/4 4 3 Structure of the Gross Sample 6 4 Questionnaires and fieldwork material 8 4.1 Questionnaires 8 4.2 Fieldwork material 8 5 Conducting the survey 11 5.1 Survey mode 11 5.2 Fieldwork timings 11 5.3 Translations 12 5.4 Panel Maintenance 14 5.5 Movers and Tracing 15 5.6 Interviewer Characteristics and Training 15 6 Fieldwork results 17 6.1 Participation on household level 17 6.2 Participation on individual level 20 6.3 Participation by types of questionnaires 21 6.4 Interview length per questionnaire 21 6.5 Consent to Record Linkage 22 6.6 (App-)Follow Up 22 7 Data preparation 24 8 Delivered data 26 © Kantar 2021 3 1 Introduction Table 1 gives an overview of the main characteristics of the first wave of the refugee boost samples M3 and M4 of 2016. While a total of 14,913 addresses were provided from the “Ausländerzentralregister” (AZR), the gross sample of M3 consisted of 4,051 households, 1,769 of whom participated in the survey between June and October 2016. This results in an adjusted response rate of 61.8 percent. From 10,850 addresses provided for M4, the gross sample consisted of 3,688 households, 1,769 of whom participated in the survey between August and December 2016. This results in an adjusted response rate of 62.2 percent. As is usually the case for boost and refreshment samples in the SOEP, only questionnaires were fielded in CAPI mode. No other modes were used in both samples. Partial unit non-response (PUNR) was at 53.7 percent (M3) and 57.3 (M4) percent respectively. Table 1: Summary fieldwork Sample M3 M4 Fieldwork period June - October August - December Mode (main questionnaires) CAPI CAPI Gross sample (hh) 4,051 3.688 Net sample (hh) 1,769 1,769 Response rate (adjusted; hh)1 61.8 62,2 Number of questionnaires Adults: 2 Youths: 0 Children: 0 Adults: 2 Youths: 0 Children: 0 Net sample (individuals) Adults: 2,351 Youths: 0 Children: 0 Adults: 2,466 Youths: 0 Children: 0 Questionnaire length (median, in minutes) Household: 18 Adult: 84 Household: 20 Adult: 76 Partial unit non-response (PUNR) 53.7 57.3 1 RR = percentage of all households with at least one household and individual interview in the gross sample (gross sample adjusted for households where the last person is deceased or the household moved abroad, is permanently untraceable or dissolved households where the last member moved into another SOEP household). © Kantar 2021 4 2 Background Samples M3/4 In order to implement an innovative sampling procedure for mapping recent migration and integration dynamics, the SOEP at DIW Berlin, the Institute or Employment Research (IAB Nuremberg), and the Research Centre of the Federal Office for Migration and Refugees (BAMF-FZ) formed a research partnership. This alliance also facilitated drawing samples for research from the Central Register of Foreign Nationals (Ausländerzentralregister, AZR). M3 is the acronym for the first top-up sample of households that represents adult refugees who entered Germany from January 1, 2013 until January 31, 2016 and applied for asylum in Germany. The sample consists of two tranches. The second tranche was necessary because in the second half of 2015, so many refugees entered Germany that this led to a gap between application for asylum and registration in the Central Register of Foreign Nationals. M4 is the acronym for the second new refugee top-up sample. It consists of two tranches as well. The first one is a household boost of the M3 sample. For the second tranche, underage children of refugee families were sampled as key informants, but only the adults in their respective households were asked to participate. The sampling frame of the Central Register of Foreign Nationals provides only basic information about foreigners in Germany, including: name, date of birth, and a registration number linked to the local recordkeeping authority. Thus, the BAMF-FZ was in charge of contacting those local recordkeeping authorities to obtain actual addresses of the refugees. As experts in the SOEP group at DIW Berlin conducted the drawing of the gross samples, we will provide some general information on the sampling procedure. A stratified multistage approach was used to draw the gross sample. ◼ Each available dataset was flagged to indicate membership in the target group of refugees entering Germany since January 1st, 2013 until January 31, 2016 who applied for asylum according to the information of the register. ◼ All datasets were linked to the local recordkeeping authority. They were the primary sampling units (PSU) in accordance with strata based on the information of the Central Register of Foreign Nationals. Local recordkeeping authorities with smaller number of refugees have been integrated into synthetic PSUs. ◼ The sampling of 130 PSU, stratification by federal state and administrative district are based on the Central Register of Foreign Nationals. ◼ The gross overall sampling of eligible registration numbers to be supplied with addresses by the local recordkeeping authority included: 80 addresses per PSU for M3 T1, 40 addresses per PSU for M3 T2, and 45 addresses per PSU for M4 T1 and T2 in each tranche. ◼ This procedure should have led to n=27,300 addresses in the overall sample. Due to a lack of cooperation by local recordkeeping authorities (in time), refugees leaving the their local recordkeeping authorities’ designated area before registering their address, and unaccompanied minor refugees excluded by the local data-keeping authorities for M4 T2, this resulted in a total of 25,763 addresses in the gross samples, M3 and M4. © Kantar 2021 5 ◼ BAMF-FZ provided Kantar Public with these addresses. In order to conduct a sufficient number of interviews, Kantar Public drew a gross sample for fieldwork: 24 addresses per PSU4 in M3 T1,6 addresses per PSU in M3 T2, 11 addresses per PSU in M4 T1, and 17 addresses per PSU in M4 T2. ◼ This procedure resulted in 7,739 addresses in gross samples M3 and M4 for the fieldwork. © Kantar 2021 12 5.3 Translations Language problems during the interviewing process form a potential major challenge for surveys with populations that recently entered the country as refugees. Although some of the deployed interviewers in M3 and M4 speak Arabic, Farsi, or Pashtu, it is generally not feasible to match interviewers with special language skills with respondents in such a large nationwide survey, particularly if in advance there is no knowledge about the respective target person’s background and skills. The solution was to use an innovative bilingual CAPI- program. Instead of using translated questionnaires on paper, which served as a reference for the interviewers and the interviewees in recent M1 and M2 samples, the translation was scripted into the CAPI such that German and another language were shown on the screen at the same time. The language was selected at the beginning of the interview. There was no way to switch to another language during the interview, which did not turn out to be a major issue. Moreover, a foreign-language hotline was set up to help interviewer and interviewee arrive at an agreement on the language for their upcoming interview and to help with all other issues and concerns regarding its nature and scope. If interviewee could not read and write in their respective language well enough, audio files were available on each screen for interviewees to listen to the questions and answers in their respective language. As this procedure was quite complex and costly, the number of languages offered to the interviewees was limited to six: English, Arabic, Farsi, Urdu, Pashto and Kurmanji. All written materials and fieldwork documents for the households were translated into these six foreign languages as well. Figure 1 and 2 show examples of this approach. Table 7: Available language versions German / English German / Arabic German / Farsi German / Pashto German / Urdu German / Kurmanji Household questionnaire       Individual + life history questionnaire       As expected, Arabic was used most frequently in almost two-thirds of all interviews (M3: 63.1 percent, M4: 64.8 percent). The results of using the different language versions are shown in Table 8. The language versions of Pashto, Urdu and Kurmanji were only used for a very limited number of interviews. In contrast, English (M3: 18.5 percent, M4: 15.4 percent) and Farsi (M3: 11.1 percent, M4: 12.9 percent) were used with the second highest regularity; with English as the number one choice when none of the provided languages were the interviewees’ native language. Table 9 shows the extent to which translations were used in interviewing. The English translation, often called upon even when the interview was conducted in German, was as expected not used or hardly used in the majority of the interviews. However, at least one fifth of the interviews required the English translation for each question. For 64 percent of the interviews in the Arabic version, which as mentioned above was used very frequently, the translation was also needed for every question. At 9 percent, the percentage of cases in which the translation was requested, but not required, is comparatively low. A very similar distribution is shown in the Farsi and Urdu versions. In Pashto and Kurmanji, a somewhat different picture emerges; here the response category “not at all” is much higher. © Kantar 2021 13 Figure 1: Screenshot bilingual CAPI program – language selection Figure 2: Screenshot bilingual CAPI program with audio files © Kantar 2021 14 Table 8: Utilization of a certain language version – individual questionnaire Individual level M3 M4 Abs. In % Abs. In % Total 2,351 100.0 2.466 100.0 German / English 436 18.5 380 15.4 German / Arabic 1,483 63.1 1.598 64.8 German / Farsi 260 11.1 317 12.9 German / Pashto 39 1.7 49 2.0 German / Urdu 55 2.3 22 0.9 German / Kurmanji 78 3.3 100 4.1 Table 9: Intensity of use of a certain language version M3 plus M4 in percent Individual level Englisch Arabic Farsi Pashto Urdu Kurmanji Every question 19.8 63,7 58,0 31,4 54,2 36,5 More than two thirds 7.0 12,0 13,7 10,5 15,3 11,9 More than half 5.9 8,0 8,9 7,0 13,9 7,5 Less than half 9.8 6,9 6,6 9,3 8,3 13,8 Not at all 57.5 9,3 12,8 41,9 8,3 30,2 In contrast to the translations, the audio files were used more rarely and called upon rather selectively in interviews. With longer texts and more demanding questions, especially if the target person had a certain weakness in reading, audio files were helpful in the interview situation. Overall the audio files proved helpful in all language versions. In addition to the questionnaires and the fieldwork materials that were available in different languages, the interviewers could call a so-called “interpreter hotline” during the process of contacting the households. Then other interviewers that are fluent in either Arabic or Farsi helped the interviewer, e.g., to explain the study background, answer questions and set up an appointment for the interview. But this service could only be used to convince respondents to participate in the study. The interview itself needed to be conducted with the bilingual questionnaires. 5.4 Panel Maintenance In samples M3 and M4, households did not receive any cash incentives or vouchers. As refugees are recipients of state benefits, cash incentives would be obliged to register and might lead to reduction of benefits. Further, qualitative surveys prior to the main study showed that cash incentives might be problematic because most © Kantar 2021 15 refugees lived within shared accommodation, which could cause enviousness and conflict between the inhabitants. Instead, interviewers were advised to bring a small gift such as local specialties. 5.5 Movers and Tracing Of the 7,739 households in the administered gross sample of M3 and M4, 24.8 percent of households were identified as movers and therefore their new addressed needed to be traced (see Table 10). The tracing of these households was successful in 18.6 percent of all cases. Most new addresses were gained by the interviewers themselves (91.3%), followed by information provided by the Postal Service (7.0%). Table 10: Movers and sources of new addresses of administered sample Household Level M3 M4 Total Abs. In % Abs. In % Abs. In % Gross Sample 4,051 100.0 3,688 100.0 7,739 100.0 Movers 1,156 28.5 760 20.6 1,916 24.8 Outcome Tracing 1,156 100.0 760 100.0 1,916 100.0 Tracing successful 202 17.5 154 20.3 356 18.6 Tracing not successful 954 82.5 606 79.7 1,560 81.4 Source 202 100.0 154 100.0 356 100.0 Interviewer 182 90.1 143 92.9 325 91.3 Postal Service 16 7.9 9 5.8 25 7.0 Participant 4 2.0 2 1.3 6 1.7 5.6 Interviewer Characteristics and Training All interviewers were part of a special interviewer staff established exclusively for the first wave of samples M3/4. Many of them can speak Arabic or Farsi. Due to the fact that many of these interviewers were specifically recruited for the refugee samples, the staff is not only notably younger in age compared to the interviewers in other SOEP samples, they are also less experienced as interviewers. Interviewer characteristics are shown in Table 11. 72 Interviewers were deployed in sample M3, 85 in sample M4. 44 Interviewers were deployed in both samples. All interviewers in samples M3 and M4 took part in an extensive, project specific one-day training event prior to fieldwork. © Kantar 2021 16 Table 11: Interviewer characteristics Interviewer Level M3 M4 Abs. In % Abs. In % Number of Interviewers 72 100.0 85 100.0 Gender Male 45 62.5 53 62.4 Female 27 37.5 32 37.6 Age 21-39 25 34.7 27 31.8 40-59 27 37.5 35 41.2 60-79 20 27.8 23 27.1 Experience with Kantar 0-4 years experience with Kantar 35 73.6 72 84.7 5-9 years 10 13.9 7 8.2 10-19 years 9 12.5 6 7.1 Experience with SOEP 0-4 years 62 86.1 78 91.8 5-9 years 5 6.9 5 5.9 10-19 years 5 6.9 2 2.4 Number of Households1 1 – 99 60 83.3 65 89.0 More than 100 12 16.7 8 11.0 1 M4:12 Interviewers that were part of the original staff for this sample did not in fact attend to any households. © Kantar 2021 17 6 Fieldwork results 6.1 Participation on household level There was no prior empirical evidence for creating a refugee sample of persons entering Germany recently for large-scale quantitative social research. Expectations about what might happen during face-to-face interviews were quite unclear. The challenge was to conduct interviews on-site in different settings with maximum flexibility. Most respondents were living independently in their own households. Some respondents were living in federal or community-organized accommodations for asylum seekers. With these accommodations, the first challenge was to obtain permission to gain access to them. Secondly, because many refugees were expected to move around a lot, at least within Germany, the challenge of encountering them at the address provided by the local recordkeeping authorities was anticipated. And thirdly, the specific background of the target population led to the assumption that most of them would not speak German well enough to be interviewed, also making it difficult for interviewers to communicate their questions. Given this situation, only a very vague estimation of the expected cooperation rate was possible in advance. Nevertheless, the first weeks of the fieldwork period clearly indicated that each possible stage of attrition would not affect this survey as much as other surveys targeting the general population in Germany. Table 12 shows the fieldwork results for samples M3 and M4. About one-third of the addresses were either not valid, or their validity was unclear. About 20 percent of the addresses were valid, but an interview could not be obtained for various reasons. In both samples, 1,769 households could be interviewed. As the SOEP standard procedural preference of interviewing all household members was not a major objective for the first wave of the two refugee samples, the number of households that were partially interviewed (i.e., at least one individual questionnaire was missing) is comparably high. However, 582 additional people in M3 and 697 in M4 were interviewed, resulting in 2,351 individual interviews in M3 and 2,466 in M4. Tables 13 and 14 show the fieldwork results by different gross sample definitions and Table 15 the different outcome rates for both samples. Only a few cases had to be excluded because the key informant was an unaccompanied refugee minor. In addition, it was seldom the case that the key informant belonged to a household in which another key informant who we had already interviewed also lived. Those cases are listed as QNDs in Tables 13 and 14. With these kinds of attrition and the low number of invalid addresses, there were a lot of addresses left over for fieldwork. And this proved to be very crucial. In M3, the interviewers could not process 27.4 percent of all addresses (gross sample I). That defines gross sample II as containing 2,941 viable addresses. After adjusting for deceased key informants and those who had moved abroad, 2,863 addresses remained (70.7 percent of gross sample I). Overall, the interviewers were able to contact 2,547 key informants, that is, 62.9 percent of gross sample I. Compared to the recent general refresher samples in the SOEP, the response rate of 61.8 percent, defined as the number of interviews divided by adjusted gross sample II, is very high. © Kantar 2021 18 In M4, the interviewers were unable to process 21.1 percent of all addresses (gross sample I). That defines gross sample II as containing 2,910 viable addresses. After adjusting for deceased key informants and those who had moved abroad, 2,845 addresses remained (77.1 percent of gross sample I). Overall, the interviewers were able to contact 2,550 key informants, that is, 69.1 percent of gross sample I. The response rate in M4 of 62.2 percent, defined as the number of interviews divided by adjusted gross sample II, is as remarkable as in M3. Table 12: Participation on household level Household level M3 M4 Total Abs. in % Abs. in % Abs. in % Gross sample for fieldwork 4,051 100.0 3,688 100.0 7,739 100.0 Unknown eligibilty 1,194 29.5 832 22.6 2,026 26.2 - Unable to reach during fieldwork period 316 7.8 295 8.0 611 7.9 - Key informant moved and unable to obtain address 878 21.7 537 14.6 1,415 18.3 Not eligible (e.g. business address, address does not exist) 232 5.7 241 6.5 473 6.1 Eligible, non-interview 856 21.1 846 22.9 1,702 22.0 - Key informant deceased or permanently living abroad 78 1.9 65 1.8 143 1.8 - Permanently physically or mentally unable / incompetent 37 0.9 24 0.7 61 0.8 - Language problems 153 3.8 158 4.3 311 4.0 - “Soft refusal” (currently not willing / capable) 372 9.2 435 11.8 807 10.4 - Permanent refusals 115 2.8 123 3.3 238 3.1 - Other (e.g. detained, refusal by refugee housing) 101 2.5 41 1.1 142 1.8 Interview (of key informant) 1,769 43.7 1,769 48.0 3,538 45.7 - Household completely interviewed (including single households) 1,291 31.9 1,035 28.1 2,326 30.1 - Household partially interviewd 478 11.8 734 19.9 1,212 15.7 © Kantar 2021 19 Table 13: Fieldwork results M3 in different gross samples Abs. in % gross sample I In % gross sample II Gross sample I (all gross addresses for fieldwork) 4.051 100,0 Non processable addresses (not attempted; anchor person moved / unable to obtain new address; QNDs) 1.110 27,4 Gross sample II (processable addresses) 2.941 72,6 Deceased and moved abroad 78 1,9 Gross sample II adjusted 2.863 70,7 100,0 Unable to reach during fieldwork period 316 7,8 11,0 Contacted processable addresses 2.547 62,9 89,0 Non Cooperation (Permanently unable / incompetent; Language problems; soft and permanent refusals) 778 19,2 27,2 Valid Interviews 1.769 43,7 61,8 Table 14: Fieldwork results M4 in different gross samples Abs. in % gross sample I In % gross sample II Gross sample I (all gross addresses for fieldwork) 3,688 100,0 Non processable addresses (not attempted; anchor person moved / unable to obtain new address; QNDs) 778 21.1 Gross sample II (processable addresses) 2,910 78.9 Deceased and moved abroad 65 1.8 Gross sample II adjusted 2,845 77.1 100,0 Unable to reach during fieldwork period 295 8.0 10.4 Contacted processable addresses 2,550 69.1 89.6 Non Cooperation (Permanently unable / incompetent; Language problems; soft and permanent refusals) 781 21.2 27.5 Valid Interviews 1,769 48.0 62.2 Table 15: Outcome rates M3 and M4 at household level M3 M4 Household level In % gross sample I In % gross sample II adjusted In % gross sample I in % gross sample II adjusted Contact rate (contacted addresses / gross sample) 62,9 89,0 69,1 89,6 Response rate (interviews / gross sample) 43,7 61,8 48,0 62,2 © Kantar 2021 20 6.2 Participation on individual level Table 16 presents participation on an individual level. Of altogether 6,394 individuals in the gross sample, 75.3 percent were successfully interviewed, while 24.7 percent either dropped out temporarily (22.4 percent) or permanently (2.3 percent). The low percentage of final dropouts is a good result, especially with regards to the second wave and the potential conversion of soft refusals and their future inclusion in the panel. The response rates on the individual level are provided in Table 17. Table 16: Participation on individual level Individual level M3 M4 Total Abs. In % Abs. In % Abs. In % Gross Sample1 3,012 100.0 3,382 100.0 6,394 100.0 Interview 2,351 78.1 2,466 72.9 4,817 75.3 Non-Interview 661 21.9 916 27.1 1,577 24.7 Temporary dropout 563 18.7 868 25.7 1,431 22.4 Non-contact 27 0.9 157 4.6 184 2.9 Temporary refusal 416 13.8 594 17.6 1,010 15.8 Temporarily physically or mentally unable/incompetent 48 1.6 27 0.8 75 1.2 Language problem 34 1.1 62 1.8 96 1.5 Other temp. 38 1.3 28 0.8 66 1.0 Final Dropout 98 3.2 48 1.4 146 2.3 Permanent Refusal 88 2.9 44 1.3 132 2.1 Permanently physically or mentally unable/incompetent 10 0.3 4 0.1 14 0.2 1 All household members of participating households. Table 17: Response rate M3 M4 Total Response Rate 78.1 72.9 75.3 One major concern for all SOEP samples are the growing partial unit non-response (PUNR) rates, which are exceptionally high for the refugee samples. PUNR in sample M3 was at 53.7 percent in this year’s first wave (Table 18) which is slightly lower than the PUNR in the sister sample M4 (57.3 percent). However, in the first wave of the two refugee samples the interviewer focused on interviewing the sampled key informant. From wave two on the SOEP standard procedural preference of interviewing all household members will become a major objective. © Kantar 2021 21 Table 18: Partial unit non response M3 M4 PUNR1 53.7 57.3 1 Share of households (number of household members > 1) with at least one missing individual questionnaire. 6.3 Participation by types of questionnaires Table 19 presents the number of interviews and response rates for each of the two fielded questionnaires. Corresponding with the number of households in the net sample, 1,769 household questionnaires were produced in both samples, resulting in an adjusted response rate of 61.8 percent (sample M3) respectively 62.3 percent (sample M4). The response rates for the individual and life history questionnaire are 78.1 percent (sample M3) respectively 72.9 percent (sample M4). Table 19: Number of interviews and response rate by questionnaire M3 M4 Gross sample1 Number of interviews Response rate Gross sample1 Number of interviews Response rate Household questionnaire 2,863 1,769 61.8 2,845 1,769 62.2 Individual questionnaire + life history questionnaire module 3,012 2,351 78.1 3,382 2,466 72.9 1 Gross sample = target population in participating households, excluding deceased, moved abroad. 6.4 Interview length per questionnaire The media interview length for key informants in both refugee samples was about 100 minutes (see Table 20). In many cases, the interview lasted three hours or more. The median interview length for other household members who completed the individual questionnaire without the household section was about 80 minutes. Further, one must add significant time for the contact phase. In many cases the interviewers had to answer questions after the interview was completed. Therefore, the survey was very demanding and time-consuming for both the interviewees and the interviewers. Table 20: Median interview length in minutes M3 M4 Household questionnaire 18 20 Individual questionnaire + life history questionnaire module 84 76