f the analysis. For example, a multiple stage model involving constructs or concepts affecting another concept or concepts that in tum affect some other concept would require a particular analytical technique like structural equation modelling.

Such models are by no means far fetched and represent the type of situation encountered in trying to understand consumer decision making processes. Often they are best handled with structural equation modelling that has its own sample size requirement irrespective of generalisability issues. Indeed too large a sample may give a wrong answer because the statistical technique may be oversensitive with large samples. This may sound a little daunting if all the resea

What makes a market research survey a good survey? There is no simple answer to this question, and it is not one aspect, but various aspects together that make for a good piece of research. Indeed, market research is very much a balancing act where the researcher often has to deal with decisions that have conflicting consequences. The researcher needs to balance out the various elements to ensure that much of what is gained on the swings is not lost on the roundabouts. This, after all, is what the research process is all about, and involves a dedicated attempt at reducing error, in the knowledge that one can never eliminate error completely. Many devote much attention to sampling issues yet paper over questionnaire issues that are often a much larger source of error. This article starts by looking at two important sampling issues and then proceeds to consider two other issues in questionnaire design that often receive scant attention.

Sampling Concerns

Whenever one is asked to undertake research, one of the first issues that tends to crop up is that of sample size. At least two aspects need to be highlighted in sample size considerations. First, are the results of the sample to be generalised to an entire population? This sounds like a trivial question and a somewhat shocked 'of course' refrain is to be expected. However, the corollary to this is, with what degree of accuracy and confidence do you want this to happen? Here lies the rub because these are researcher or management decisions and on this basis one can justify a wide range of sample sizes! In fact, more often than not, statistical considerations for sample size determination are of secondary importance.

A more relevant question to ask for sample size determination is perhaps what is the intended use of the findings? Will they be used to make a critical decision; as part of a PR exercise; or is it just a case of a 'nice to know' situation? Only the first objective is likely to require a representative sample. However, irrespective of sample size it is very desirable that a random sample is collected as this allows for statistical analysis of the data. Any random sample, even if the sample is not generalisable, can be analysed statistically. Moreover, there is nothing wrong with findings from a sample that are not generalisable as long as the conclusions are clearly bounded by the limitations of the sample.

A second aspect concerning sample size that is useful to consider relates to what type of analysis it is intended to undertake with the data collected, so as to answer your research question or questions. The size of the sample may seriously limit what analysis is possible to undertake. Different research issues or models will demand different statistical analytical tools and different sample size requirements to ensure the robustness of the analysis. For example, a multiple stage model involving constructs or concepts affecting another concept or concepts that in tum affect some other concept would require a particular analytical technique like structural equation modelling.

Such models are by no means far fetched and represent the type of situation encountered in trying to understand consumer decision making processes. Often they are best handled with structural equation modelling that has its own sample size requirement irrespective of generalisability issues. Indeed too large a sample may give a wrong answer because the statistical technique may be oversensitive with large samples. This may sound a little daunting if all the resear

nnaire issues that are often a much larger source of error. This article starts by looking at two important sampling issues and then proceeds to consider two other issues in questionnaire design that often receive scant attention.

Sampling Concerns

Whenever one is asked to undertake research, one of the first issues that tends to crop up is that of sample size. At least two aspects need to be highlighted in sample size considerations. First, are the results of the sample to be generalised to an entire population? This sounds like a trivial question and a somewhat shocked 'of course' refrain is to be expected. However, the corollary to this is, with what degree of accuracy and confidence do you want this to happen? Here lies the rub because these are researcher or management decisions and on this basis one can justify a wide range of sample sizes! In fact, more often than not, statistical considerations for sample size determination are of secondary importance.

A more relevant question to ask for sample size determination is perhaps what is the intended use of the findings? Will they be used to make a critical decision; as part of a PR exercise; or is it just a case of a 'nice to know' situation? Only the first objective is likely to require a representative sample. However, irrespective of sample size it is very desirable that a random sample is collected as this allows for statistical analysis of the data. Any random sample, even if the sample is not generalisable, can be analysed statistically. Moreover, there is nothing wrong with findings from a sample that are not generalisable as long as the conclusions are clearly bounded by the limitations of the sample.

A second aspect concerning sample size that is useful to consider relates to what type of analysis it is intended to undertake with the data collected, so as to answer your research question or questions. The size of the sample may seriously limit what analysis is possible to undertake. Different research issues or models will demand different statistical analytical tools and different sample size requirements to ensure the robustness of the analysis. For example, a multiple stage model involving constructs or concepts affecting another concept or concepts that in tum affect some other concept would require a particular analytical technique like structural equation modelling.

Such models are by no means far fetched and represent the type of situation encountered in trying to understand consumer decision making processes. Often they are best handled with structural equation modelling that has its own sample size requirement irrespective of generalisability issues. Indeed too large a sample may give a wrong answer because the statistical technique may be oversensitive with large samples. This may sound a little daunting if all the resea

t this to happen? Here lies the rub because these are researcher or management decisions and on this basis one can justify a wide range of sample sizes! In fact, more often than not, statistical considerations for sample size determination are of secondary importance.

A more relevant question to ask for sample size determination is perhaps what is the intended use of the findings? Will they be used to make a critical decision; as part of a PR exercise; or is it just a case of a 'nice to know' situation? Only the first objective is likely to require a representative sample. However, irrespective of sample size it is very desirable that a random sample is collected as this allows for statistical analysis of the data. Any random sample, even if the sample is not generalisable, can be analysed statistically. Moreover, there is nothing wrong with findings from a sample that are not generalisable as long as the conclusions are clearly bounded by the limitations of the sample.

A second aspect concerning sample size that is useful to consider relates to what type of analysis it is intended to undertake with the data collected, so as to answer your research question or questions. The size of the sample may seriously limit what analysis is possible to undertake. Different research issues or models will demand different statistical analytical tools and different sample size requirements to ensure the robustness of the analysis. For example, a multiple stage model involving constructs or concepts affecting another concept or concepts that in tum affect some other concept would require a particular analytical technique like structural equation modelling.

Such models are by no means far fetched and represent the type of situation encountered in trying to understand consumer decision making processes. Often they are best handled with structural equation modelling that has its own sample size requirement irrespective of generalisability issues. Indeed too large a sample may give a wrong answer because the statistical technique may be oversensitive with large samples. This may sound a little daunting if all the resea

f the data. Any random sample, even if the sample is not generalisable, can be analysed statistically. Moreover, there is nothing wrong with findings from a sample that are not generalisable as long as the conclusions are clearly bounded by the limitations of the sample.

A second aspect concerning sample size that is useful to consider relates to what type of analysis it is intended to undertake with the data collected, so as to answer your research question or questions. The size of the sample may seriously limit what analysis is possible to undertake. Different research issues or models will demand different statistical analytical tools and different sample size requirements to ensure the robustness of the analysis. For example, a multiple stage model involving constructs or concepts affecting another concept or concepts that in tum affect some other concept would require a particular analytical technique like structural equation modelling.

Such models are by no means far fetched and represent the type of situation encountered in trying to understand consumer decision making processes. Often they are best handled with structural equation modelling that has its own sample size requirement irrespective of generalisability issues. Indeed too large a sample may give a wrong answer because the statistical technique may be oversensitive with large samples. This may sound a little daunting if all the resea

f the analysis. For example, a multiple stage model involving constructs or concepts affecting another concept or concepts that in tum affect some other concept would require a particular analytical technique like structural equation modelling.

Such models are by no means far fetched and represent the type of situation encountered in trying to understand consumer decision making processes. Often they are best handled with structural equation modelling that has its own sample size requirement irrespective of generalisability issues. Indeed too large a sample may give a wrong answer because the statistical technique may be oversensitive with large samples. This may sound a little daunting if all the research that one has come across consists of percentages. But, there is more to good research than percentages. Indeed, percentage type analysis and results are often the outcomes of a decision the researcher would have made about the data collection employed in the questionnaire used.

Pitfalls of Questionnaire Development

There is no doubt that sample size has its importance in the research process but there is much more to be concerned about besides issues of sample size. Let me start by dispelling one myth that always sends a shiver up my spine. This is the idea that anybody can sit down and write a good questionnaire. Yes of course anybody can put together a questionnaire and there is nothing stopping anyone using it either. The emphasis in my statement is however is on a good questionnaire. Since this is not the place to go into the intricacies of questionnaire development, I will only underline two aspects to show how much is often overlooked.

I will start by first highlighting how one goes about seeking to capture a concept when designing a questionnaire. What a researcher is often trying to do when conducting market or management research is to capture something that resides in the respondent's mind or black box as it is sometimes called. It is by no means an easy feat and is often not likely to be achieved by simply asking a single question. A useful analogy is of the lecturer trying to capture student knowledge about a subject. A lecturer would normally seek to determine student knowledge by asking more than one question to capture the knowledge concept. If one asks a student just one question one may have hit the one area that the student just did not study. However, if one asks a couple of questions on different aspects of knowledge the student was supposed to have learned and it turns up dry then one is far safer in drawing conclusions about the student's knowledge. An analogous multi-question procedure is similarly worthwhile to pursue in questionnaire development to capture concepts like service quality, loyalty, etc., that are similarly resident in the respondent's black box. Therefore a basic principle in questionnaire design is for the researcher to ask a battery of questions to capture each of the intended concepts.

A related issue concerns the type of data that is collected by each question in a questionnaire or research instrument, as it is often called. In simple terms it is often sensible to avoid the yes/no type of questions as these are generally simplistic, not good at capturing constructs and limit what statistical analysis and testing one can or cannot undertake. Wherever possible it is useful to use scales as these allow for more capture of variance, wider statistical analysis and testing and ultimately more useful conclusions.

A second important aspect to do with ques