Surveys

A survey is a method of gathering information from a number of individuals, known as a sample, in order to learn something about the larger population from which the sample is drawn. Although surveys come in many forms, and serve a variety of purposes, they do share certain characteristics. In order for the objectives of a survey to be met, the results must reliably project on the larger public, from which the sample is drawn. A sample can be scientifically chosen so that each individual in a population has a known chance of selection. This ensures that a sample is not selected haphazardly or uses only those eager to participate.

The sample size for a survey will depend on the degree of reliability necessary and how the results are to be used. A properly selected sample should be able to reflect the various characteristics of a total population within a very small margin for error. There are many surveys that study the total adult population but many others that focus on selected populations: employees, academics, industry experts, computers users, or customers that use a particular product or service.

Some surveys focus on opinion and attitudes while others are more concerned with collecting factual information. Many surveys combine questions of both types. A respondent could be asked what they heard or read about an issue, what they know about it, their opinion, how strongly they feel and why, interest in the issue, past experiences with it, and also certain demographic information which will help the survey analyst classify the responses (such as age, sex, marital status, occupation, and place of residence). Questions can be open ended (“What does that make you think of?”) or closed (“Do you agree or disagree?”); they may ask the respondent to rate a product or a service on some kind of scale; they may ask for a ranking of various alternatives. The questionnaire could be very brief – a few questions taking no longer than five minutes, or it could take a demanding hour or more of a respondent’s time. A survey is usually rooted in situations where an individual or institution is confronted with an information need and no existing data will suffice. Once the information need has been identified and a determination made that existing data is inadequate, objectives are laid out for the investigation. These objectives should remain as specific, clear cut and unambiguous as possible.

Designing the questionnaire is a critical stage of the survey development process. The questionnaire links the information need to the realized measurement. Scaling techniques used for measurement can be comparative or non-comparative. If proper care and attention is not placed on clearly defining concepts and unambiguously phrasing questions, the resulting data is apt to contain serious biases. Questions used in surveys must be refined to minimize interpretation problems and thus reduce measurement error. If the respondent is unable to understand a question or fails to comprehend the question the way it was intended then the data is neither valid nor reliable. A question must use language in a way that makes the intended observer’s meaning behind that question obvious. In order to do this a survey must reflect an understanding of the population being sampled. For example, many people do not distinguish between robbery which requires the theft to be in the immediate presence of the victim and burglary which can involve breaking and entering without any confrontation. The National Crime Survey, done by the Bureau of the Census, does not even mention the word “robbery” when it asks questions about robbery victimization. Rather they ask several questions that use universally understood phrases, consistent with the operational definition of robbery, that when used together are able to capture the desired responses.

Keeping response errors and biases to a minimum factor heavily in designing a survey. How questions are interpreted by respondents must be carefully considered. Also, you need to consider the length of the survey. If a questionnaire is too long than it can be burdensome to the respondent, inducing respondent fatigue that leads to response errors, refusals, incomplete questionnaires, and can contribute to higher non-response rates in subsequent surveys involving the same respondent. Memory plays an important role when surveys deal with past events. For most people the greater the demand a question places on memory, such as being asked to recall trivial details occurring on any given random date, the less accurate the responses and therefore the less reliable the survey data that is collected.

An appropriate choice of reference period should be made so that a respondent is not forced to report on events that happened too long ago. Grouping similar questions together can reduce the cognitive burden a survey places on a respondent. The validity of a survey can be put in jeopardy if questions are too sensitive, if they may prejudice the respondent, if they unduly invade a respondent’s privacy, and if the information that is sought is too difficult for a willing respondent to provide.

Strengths:
  • Surveys are relatively inexpensive (especially self-administered surveys).
  • Surveys are useful in describing the characteristics of a large population. No other method of observation can provide this general capability.
  • They can be administered from remote locations using mail, email or telephone.
  • Consequently, very large samples are feasible, making the results statistically significant even when analyzing multiple variables.
  • Many questions can be asked about a given topic giving considerable flexibility to the analysis.
  • There is flexibility at the creation phase in deciding how the questions will be administered: as face-to-face interviews, by telephone, as group administered written or oral survey, or by electronic means.
  • Standardized questions make measurement more precise by enforcing uniform definitions upon the participants.
  • Standardization ensures that similar data can be collected from groups then interpreted comparatively (between-group study).
  • Usually, high reliability is easy to obtain–by presenting all subjects with a standardized stimulus, observer subjectivity is greatly eliminated.
Weaknesses:
  • A methodology relying on standardization forces the researcher to develop questions general enough to be minimally appropriate for all respondents, possibly missing what is most appropriate to many respondents.
  • Surveys are inflexible in that they require the initial study design (the tool and administration of the tool) to remain unchanged throughout the data collection.
  • The researcher must ensure that a large number of the selected sample will reply.
  • It may be hard for participants to recall information or to tell the truth about a controversial question.
  • As opposed to direct observation, survey research (excluding some interview approaches) can seldom deal with “context.
Advantages of Survey Method:
    • As compared to other methods (direct observation, experimentation) survey yield a broader range of information. Surveys are effective to produce information on socio-economic characteristics, attitudes, opinions, motives etc., and to gather information for planning product features, advertising media, sales promotion, channels of distribution and other marketing variables.
    • Questioning is usually faster and cheaper that Observation.
    • Questions are simple to administer.
    • Data is reliable.
    • The variability of results is reduced.
    • It is relatively simple to analyze, quote and interrelate the date obtained by survey method.
Disadvantages of Survey Method:
    • Unwillingness of respondents to provide information – This requires salesmanship on the part of the interviewer. The interviewer may assure that the information will be kept secret or apply the technique of offering some presents.
    • Inability of the respondents to provide information – This may be due to

a. Lack of Knowledge
b. Lapse of memory
c. Inability to identify their motives and provide “reasons why?” for their actions

    • Human Biases of the respondents are there, for e.g., “Ego”
    • Systematic difficulties are there – it is difficult, if not impossible, to state a given question in such  a way that it will mean exactly same thing to each respondent. Similarly two different wordings of the same question will frequently produce quite different results.
How to overcome the limitations of Survey Method
  • Careful framing and phrasing of questions.
  • Careful control of data gathering by employing specially trained investigators who will observe carefully report on subtle reactions of persons interviewed
  • Cautious interpretations by a clear recognition of the limitations of the data and understating of what exactly the data represents. This is especially true of responses to questions like – “What price would you be willing to pay for this product?”
  • Looking at facts in relative rather than absolute terms. For eg – A survey by a dentist team showed that the number of families in the middle income group used toothpaste taken by itself in the absolute sense, the results of the survey are in some doubt. Even though the individual group readings shall differ say for eg: for upper income group families it could be 90 %.