Survey use in ethnographic studies

Are you a qualitative or quantitative researcher?  In academia we are typically expected to adopt one or the other of these two camps, and in so doing we get swept up in the contentious debate as to which approach is best.  The practical researcher should become skilled in qualitative and quantitative approaches, recognizing that both have their strengths and weaknesses.  In fact, combining both qualitative and quantitative methods on a project can yield rich results.

My research project on Berlin Starbucks cafes made use of mixed methods.  For that project my goal was to understand how the Starbucks baristas in Berlin, Germany made sense of, utilized, and modified the U.S. American-style customer service protocols that they were required to use.  I carried out the research in three distinct phases. I began with non-participant observation in two of the Berlin Starbucks cafes.  Next, I did in-depth, one-to-one interviews with a selection of the Berlin baristas and managers.  Finally, I created a survey and distributed it to all of the Starbucks baristas in Berlin.  (At the time of this project, there were only seven Starbucks cafes in Berlin, employing about 80 baristas.  Now I believe there are 25 cafes, and I can only guess that they must have at least a couple hundred baristas.)

While I considered my project to be primarily ethnographic in nature, the survey component of the research helped me to test the nature and distribution of the themes and concepts that I had identified in the first two phases.  Specifically, my surveys measured to what extent baristas felt comfortable with the customer service procedures that had been described to me in the interviews as especially difficult, stressful, or American/non-German.

Besides the results of the study, which were very interesting (if I do say so myself), I gleaned some practical knowledge and tips from this experience:

• Because the surveys were filled out by the baristas during their breaks, they had to be short enough to complete in 15 minutes.  (How much time will your respondents realistically have to complete your survey?)

• My original survey draft was written in English, translated into German, and then back-translated by a second party into English.  This was done to check the accuracy of the first translation and to fix any incorrect or ambiguous concepts, grammatical constructions and/or vocabulary. (Do the concepts that you are testing translate into the target language and/or culture?  Do your questions make sense and/or mean what you intend them to mean in the target language and/or culture? )

• I did trial runs of the German survey with 10 baristas, who kindly gave both written and oral feedback on the questions, critiquing clarity, vocabulary, jargon, etc. More modifications to the survey were made based on this feedback. (Do the concepts that you are testing make sense to the target community?  Are you using the correct terms and/or jargon for experts in that community?)

• The Starbucks store managers distributed the surveys to the baristas.  The baristas were asked to fill the surveys out on the premises, as I thought this would increase the likelihood of their being completed.  However, this could undoubtedly have influenced the baristas’ answers, since their privacy at work was limited.  (Ideally, what sort of environment should respondents be in when they are answering the questions?  Will respondents experience greater anonymity and/or privacy with a paper copy of the survey, or an electronic one?)

For more articles on mixed methods work done in the social sciences and other fields see the Journal of Mixed Methods Research.