Research for Marketing and Communications
DISCUSSION OF SCOPE OF SURVEY
AND SAMPLE SIZE DETERMINATION
Often in the initial stages of a research project the scope of the investigation is kept wide, then focused down as particular opportunities or issues are identified. For instance, in the initial stages of a project to determine the make-up of consumers for say, television sets, the scope of any survey is kept broad, and likely to include a nationally representative sample of individuals or households. Once the required information is obtained, the investigation can become more narrow or focused, determining the profile of consumers of a specific brand, for example, or the buying behavior of particular market segments (e.g. college educated people, small town dwellers, etc.) and people in specific lifestage groups (e.g. young people starting families in the suburbs). Subsegments, then, can be broken-out for comparison if the samples are sufficient for analysis.
Subsequent research can be planned with key market segments to address specific issues (e.g. design features, advertising, product image, etc.).
Generally, samples of 1,000 are well regarded for national surveys. (The layman, however, might be critical of samples of 1,000 or 2,000 considering an adult population of 180 million or so particularly when their favorite TV show is threatened with cancellation because of poor ratings.) The key to good research, however, is not sample size so much as it is getting representative samples. Even if we survey 100,000 people and they are not representative of our target market (let's say they are all university students and we are investigating washing detergent purchases) then sample size means nothing. Therefore, in research investigations, whenever possible, use random sampling procedures with controls in place to insure our target market is properly represented.
Let's take a look at sample sizes from a statistical reliability point of view. In the following example the statistics were obtained from Table 1: The Accuracy of Survey Data, Single Samples. With a sample of 100 the 90% level of confidence of any observed percentage is within +/- 8.2%. As the sample increases, the precision increases and the range of error gets smaller:
Thus, when we measure a 49% approval rate, say, with a sample of 1,000 people we can be 90% sure if we do the survey again, the results will be within 2.6 points of the 49%, or between 46.4% and 51.6%. We improve our precision quite a bit by going from 1,000 to 2,000 sample size (0.8 points), but only marginally by going from 2,000 to 3,000 (0.3 points). The selection of sample size should depend upon the consequences of coming to a wrong conclusion, and the cost of adding sample size.
When we start looking into demographic segments within the sample, the sample sizes get smaller and, hence, the statistical reliability again gets called into question. It is unlikely that any segment we choose will have fewer than 15% of the population, or a sample of 300 people. Thus sub-group analysis with samples of 300 would be within 4.7% at the 90% confidence level, and comparisons between two subgroups of 300 would be within +/-6.7 percentage points (and could be less) for any statistic generated (See Table 2: Accuracy of Samples for Two Populations.)