Research methods. Uses and limitations of questionnaires, interviews, and case studies

Table of Contents

1 INTRODUCTION

2 QUESTIONNAIRE
2.1 STRENGTHS
2.2 LIMITATIONS

3 INTERVIEW
3.1 STRENGTHS
3.2 LIMITATIONS

4 CASE STUDY
4.1 STRENGTHS
4.2 LIMITATIONS

5 CONCLUSION

6 BIBLIOGRAPHY

1 Introduction

At the beginning of any type of research, it is important for the researcher to determine the most appropriate methodology to carry out the study. The Dictionary of Sociology (1998) defines methodology as a “the methods and general approach to empirical research of a particular discipline”. It is implied that various methods exist to approach a particular research problem, and the researcher should give his or her own set of methods considerable thought.

While factors such as time and costs certainly play an important part in deciding how to approach a particular research problem, the subject of the research itself should ultimately determine the methods used. A good researcher will evaluate all available options prior to making a decision as to which methods to adapt in the light of being the most useful for the study at hand.

Scandura and Williams (2002) note that the impact of management studies often depends ‘upon the appropriateness of the research methods chosen’. This further highlights the importance the researcher needs to place at the selection of the right approach if the end-result is expected to be valuable and meaningful from a management perspective.

When the most appropriate research method - or a mix of various methods - has been established, it is time to start what Gilbert (2001) calls detective work: “Social research involves detective work. You begin with a problem and then ask a number of questions about it, such as ‘what?’, ‘who?’, ‘where?’, ‘when?’, ‘how?’ and ‘why?’”.

A variety of quantitative and qualitative research methods are available to the researcher, ranging from interviews, questionnaires, observation, experiments, to case studies. This paper will focus on three of the above methods, namely questionnaires, interviews, and case studies, in chapters two, three, and four respectively. Each chapter will give a brief introduction about the method, and then highlight the main strengths and limitations of each approach. For the purpose of this paper, induction is defined as a data-driven approach to research, while deduction is seen as theory-driven.^[1] As such, the notion of data-driven versus theory-driven will also be explored briefly for each method. Lastly, chapter five will offer a conclusion and the main findings of this paper.

2 Questionnaire

A questionnaire is essentially a structured technique for collecting primary data. It is generally a series of written questions for which the respondents has to provide the answers (Bell 1999). While authors such as Kervin (1999) offer a very narrow definition of questionnaires (whereby the person answering the questions actually records his or her own answers), deVaus (1996) sees a questionnaire in a much wider context (namely as a technique in which various persons are asked to answer the same set of questions).

Care has to be taken in creating a questionnaire; Oppenheim (1996) comments that ‘the ability to write plain English will help’, but that that will not be sufficient. If a questionnaire is well designed, it will motivate the respondents to give accurate and complete information; as such, it should provide reliable and relevant data in return.

A questionnaire can serve as an inductive method with the aim to formulate new theory, where open-ended questions are used to ‘explore a substantive area’ (Gill & Johnson 2001). However, other researchers may use a questionnaire as a means of collecting reliable data in a rather deductive approach in order to test existing theory. As will also be shown in the case of the other two research methods discussed in this paper, a questionnaire thus allows the researcher to adopt either an inductive or a deductive approach, or even a combination of these two.

When developing the actual questionnaire, QuickMBA (Questionnaire Design 2002) explains the three major question-types available to the researcher, namely:

- Open-ended: They have the advantage of offering a wide range of responses that help to capture the flavour of people’s answers, while not influencing the outcome of the question by pre-determining possible responses. On the down-side, answers are often difficult to evaluate and tend to vary in clarity and depth;
- Dichotomous (closed-ended): They are used for questions with two possible opposing outcomes, for example ‘Yes’ and ‘No’. They tend to be easier to answer and require less effort when interpreting the results - they are directly comparable to answers by other respondents; and
- Multichotomous (closed-ended): Questions of this type offer a range of possible answers, similar to a multiple-choice test. Again, they tend to be easier on the respondent and equally on the questionnaire-interpreter later on.

2.1 Strengths

If administered properly, questionnaires can prove to be an excellent method to obtain quantitative data about people’s attitudes, values, experiences, and past behaviour (Bell 1999).

Questionnaires allow the researcher to gather a significant amount of data at relatively little cost. Questionnaire distributed by post can be posted to the target group, and the latter can choose to answer whenever it is most convenient for them (Gilbert 2001). Email acts as another delivery channel, and can reduce costs even further.

Of the two main types of questionnaires, descriptive and explanatory, questionnaires allow the researcher to gather data either to explain different phenomena or to explain cause-and-effect relationships between different variables respectively (Gilbert 2001).

2.2 Limitations

One of the main drawbacks of questionnaires, especially ones served through the post or electronic mail, is the often low response rate (Bell 1999). Gilbert (2001) reports that response rates for postal questionnaires can be as low as 20%. Incorrectly or illegibly filled out questionnaires, or even missing answers, will inevitably influence the quality of the data obtained, and have the potential to further lower the number of useable questionnaires.

Questionnaires do not offer the researcher the opportunity to follow up ideas and clarify issues - one of the main strengths of interviews, as chapter three will show. Consideration needs to be given to all aspects of the questionnaire, from design to selecting the correct target-group, in order to obtain the maximum amount of reliable and valuable data (Bell 1999).

Saunders et. al. (2001) notes that questionnaires are not particularly suited for research that requires a significant amount of open-ended questions, for which a (semi-structured) interview might be a better method to pursue.

Saunders et. al. (2001) also describe the limitations of questionnaires with regards to the expected outcome, which might for example highlight trends or attitudes, but will fail to explain the underlying reasons for the outcome. A ‘multi-method’ approach, whereas the researcher combines questionnaires with, for instance, interviews to explain the results, is therefore proposed.

Another limitation of questionnaires can be found in the structure of the method itself. Fixed-choice questionnaires generally assume an un-stated general knowledge of the topic being investigated, and force the respondent to answer questions that he or she might be ignorant of, have a different understanding of based on personal perception, or which are influenced by exogenous factors such as education, culture, age, or societal status (Hyman 1955). A questionnaire has no means of correcting this; the outcome might thus be slightly biased at best, or plainly misleading.

[...]

^[1] It is understood that the terms induction and deduction imply a significantly more than simply data-driven or theory-driven approaches to research. A deeper analysis of the above, however, was not seen as appropriate for inclusion in this paper, which deals primarily with the strenths and limitations of the mentioned research methods. For a more thorough discussion of induction and deduction, see Gill and Johnson (2002), chapter three.