Tom Minsel, Ph.D., Head of Research and Data Science
Recently, Trone Research + Consulting (TR+C) was tasked with conducting a quantitative study to help a client better understand where and how they should invest in their facilities. This client had a sizable customer base of more than 1.1 million customers, comprised of both individual consumers and businesses.
Data Collection
Sample data was obtained by emailing invitations to our client's customer database. As is often the case with old lists, there proved to be some difficulty in reaching customers. Some email addresses were no longer valid. Other emails were being delivered to spam or junk filters. Plus, since many of our client’s customers were businesses, sometimes the email address on file was for the person who had been in contact with our client to make arrangements but not necessarily the same person who had used our client’s services. After many interventions, we ended up with 296 completed surveys.
While consulting with our client’s executive board about the survey results, a pressing question was raised. Is a sample quantity of 296 large enough to gather reliable information on the entire client base and make important investment decisions?
Can less than 300 people represent more than a million customers?
In this case, the answer is, yes.
Statistical Analysis
Two inferential statistics useful in answering this question are the level of confidence and the margin of error. In survey research, these give us a range within which a measured observation would fall if the question used to obtain the measure was asked across 100 samples. For example, suppose we obtain 95% confidence with a margin of error of +/-5 and observe that 75% of respondents to our survey find Brand X appealing. What this tells us is that the actual appeal of Brand X lies between 70% and 80%. Furthermore, we’re 95% confident that this is the case. In other words, if we conducted our study 100 times, in 95 of those 100 times, we would observe values between 70% and 80%. The other 5% of the time, we’d observe data outside this range.
The following table details margins of error at various levels of confidence common to market research given a sample size of n=296 pulled from a universe size of 1,136,898.
| Universal Size = 1,136,898 | |
|---|---|
| Sample Size n=296 | Margin of error +/- |
| 95% confidence | 5.70 |
| 90% confidence | 4.78 |
| 85% confidence | 4.18 |
Per the table above, a sample of n=296 will yield 90% confidence with a margin of error of +/-4.78 which, in TR+C’s judgment, is generally acceptable for market research. Looking at 95% confidence, we observe a margin of error of +/-5.70. This is also relatively decent for most market research but the acceptable margin of error varies by industry. If we were conducting clinical trialsresearch, for example, we would desire greater precision.
To quote other research professionals:
- The margin of error in social science research generally ranges from 3% to 7% and is closely related to sample size. (National Institutes of Health)
- In general, the precision of an estimate is related to the square root of the sample size—in other words, to double the precision, the sample size must be quadrupled. As a general rule, sample sizes of 200 to 300 respondents provide an acceptable margin of error and fall before the point of diminishing returns. (Kevin Lyons, Lipman Hearne)
Ultimately, it is up to you to determine the margin of error with which you can live. This decision is affected by several factors that need to be considered, including the desired level of precision, feasibility and the cost of sample as well as the investment of time.
Tom Minsel, Ph.D., of Trone Research + Consulting, works with a wide array of clients, bringing diversified experience in data-driven business strategy, database analysis and marketing, predictive modeling and research design.
Have a business or marketing challenge or question?
Contact us.
336.819.6933 kness@troneresearch.com
© 2022Trone Research and Consulting, LLC.All rights reserved.
Privacy PolicyWebsite design by Trone Brand Energy.
FAQs
How large should a sample size be for research? ›
A good maximum sample size is usually around 10% of the population, as long as this does not exceed 1000. For example, in a population of 5000, 10% would be 500. In a population of 200,000, 10% would be 20,000.
Why is 30 the ideal sample size? ›A sample size of 30 often increases the confidence interval of your population data set enough to warrant assertions against your findings. 4 The higher your sample size, the more likely the sample will be representative of your population set.
Is sample size of 20 enough? ›Sample size guidelines suggested a range between 20 and 30 interviews to be adequate (Creswell, 1998).
When sample size is less than 30 can be used? ›The T-distribution
To compute a confidence interval for a mean when the sample size is less than 30, one should use an appropriate “t-score” instead a Z score. However, the appropriate t-score will depend on the sample size and “degrees of freedom.”
Summary: 40 participants is an appropriate number for most quantitative studies, but there are cases where you can recruit fewer users.
When would a sample size greater than 30 be recommended? ›Sample size equal to or greater than 30 are required for the central limit theorem to hold true. A sufficiently large sample can predict the parameters of a population such as the mean and standard deviation.
What is the rule of 30 research? ›“A minimum of 30 observations is sufficient to conduct significant statistics.” This is open to many interpretations of which the most fallible one is that the sample size of 30 is enough to trust your confidence interval.
Is 30 a good sample size for quantitative research? ›Although sample size between 30 and 500 at 5% confidence level is generally sufficient for many researchers (Altunışık et al., 2004, s.
When sample size is 30 or less than 30 which sample test is used? ›Z-tests are closely related to t-tests, but t-tests are best performed when the data consists of a small sample size, i.e., less than 30. Also, t-tests assume the standard deviation is unknown, while z-tests assume it is known.
How much of a sample can you legally use? ›You CANNOT sample music without permission, no matter how short or long the sample is. Copyright is copyright. And if the sample is recognizable (hell, even if it isn't recognizable), you're using another person's intellectual property in order to construct or enhance your own.
How do you determine a reasonable sample size? ›
- Define population size or number of people.
- Designate your margin of error.
- Determine your confidence level.
- Predict expected variance.
- Finalize your sample size.
Sample Size for Qualitative Studies
One review identified that samples of 20 and 30 (and multiples of 10) were most common (Mason, 2010), with 25-30 being a typical recommendation (Dworkin, 2012).
Some researchers do, however, support a rule of thumb when using the sample size. For example, in regression analysis, many researchers say that there should be at least 10 observations per variable. If we are using three independent variables, then a clear rule would be to have a minimum sample size of 30.
What is a small sample size in research? ›Although one researcher's “small” is another's large, when I refer to small sample sizes I mean studies that have typically between 5 and 30 users total—a size very common in usability studies.
What happens if sample size is too small? ›Too small a sample may prevent the findings from being extrapolated, whereas too large a sample may amplify the detection of differences, emphasizing statistical differences that are not clinically relevant.
What is a good population size for a qualitative study? ›Others state that qualitative sample sizes of 20-30 are typically (pp. 56) conducted by researchers to establish data saturation using a grounded theory approach to qualitative inquiry (Creswell, 1998).
Is 50 a good sample size for qualitative research? ›While some experts in qualitative research avoid the topic of “how many” interviews “are enough,” there is indeed variability in what is suggested as a minimum. An extremely large number of articles, book chapters, and books recommend guidance and suggest anywhere from 5 to 50 participants as adequate.
What is the rule of thumb sample size? ›While determining sample size, it is usually recommended to include 20 to 30% of the population as a sample size in the form of a rule of thumb. If you take this much sample, it is usually acceptable.
What is the law of large numbers sample size 30? ›If the Sample Size < 30 , the shape of Sample Mean Distribution will be matching the shape of Population Distribution. If the Sample Size ≥ 30 , the shape of Sample Mean Distribution will be Normally distributed, regardless what the shape of Population Distribution is.
Is 30 a large enough sample size? ›In practice, some statisticians say that a sample size of 30 is large enough when the population distribution is roughly bell-shaped. Others recommend a sample size of at least 40.
What is the rule of 3 in sampling? ›
In statistical analysis, the rule of three states that if a certain event did not occur in a sample with n subjects, the interval from 0 to 3/n is a 95% confidence interval for the rate of occurrences in the population. When n is greater than 30, this is a good approximation of results from more sensitive tests.
What is Hanley's rule? ›A simple rule termed the 'rule of threes' has been proposed such that if no events are observed in a group, then the upper confidence interval limit for the number of events is three, and for the risk (in a sample of size N) is 3/N (Hanley 1983).
Is it okay to have 30 respondents in research? ›Academia tells us that 30 seems to be an ideal sample size for the most comprehensive view of an issue, but studies with as few as 10 participants can yield fruitful and applicable results (recruiting excellence is even more important here!).
Is 40 participants enough for qualitative research? ›The 2nd quartile (see table below) shows us the average values found by Marshall et al. (2012): 23 qualitative interviews for single case studies and 40 for multiple case studies. 24 interviewees for single case studies and 39 for multiple case studies.
Is 50 enough for quantitative research? ›A rule-of-thumb is that, for small populations (<500), you select at least 50% for the sample. For large populations (>5000), you select 17-27%. If the population exceeds 250.000, the required sample size hardly increases (between 1060-1840 observations).
Is a sample size of 30 too small? ›This is not a problem if the sample size is 30 or greater because of the central limit theorem. However, if the sample is small (<30) , we have to adjust and use a t-value instead of a Z score in order to account for the smaller sample size and using the sample SD.
What is a small sample size for independent t-test? ›A priori Sample Size for Independent Samples t-tests
Number 1 is t-test for the difference between two independent means or the independent samples t-test. It tells us that a small effect size is 0.20, a medium effect size is 0.50, and a large effect size is 0.80.
The Student's t-test is widely used when the sample size is reasonably small (less than approximately 30). In these cases the sample distribution of the mean is known to follow a t-distribution.
Is a sample size of 30 large enough? ›In practice, some statisticians say that a sample size of 30 is large enough when the population distribution is roughly bell-shaped. Others recommend a sample size of at least 40.
Is 15 a large enough sample size? ›You have a symmetric distribution or unimodal distribution without outliers: a sample size of 15 is “large enough.” You have a moderately skewed distribution, that's unimodal without outliers; If your sample size is between 16 and 40, it's “large enough.”
What is a good large sample size? ›
The larger the sample size, the more accurate the average values will be. Larger sample sizes also help researchers identify outliers in data and provide smaller margins of error.