Market Researh-In search of quality control

At the Asia-Pacific offices of advertising research specialist Millward Brown, a team of four people do nothing else but check the quality of interviews Millward Brown buys from its suppliers. The reliability of market research information varies enormously throughout the region, and buyers have to tread carefully. "We have put this in in the last 12 months, and it will be more than four people if it doesn't change," says Millward Brown's CEO of Asia-Pacific, Andreas Sperling. He is not optimistic: "It's not getting better." The validity of market research hinges on interviewers speaking to enough people from the target audience the research buyer is interested in. Persuading people to be interviewed, however, is not always easy, especially given the intense pressure market research companies are under to keep costs low and timeframes tight. What is more, rigorous quality controls are not yet ubiquitous in a region where the research industry is still developing. Standards have been known to slip. For a buyer, this means there's a risk of misleading results if the job has been rushed or carelessly carried out. On occasion, interviewers have saved even more time by making up the answers themselves. Two years ago, 19 interviewers were hauled up on charges of fraud after they were discovered fabricating results in a survey for the Hong Kong Tourist Board. "That's not just a one-off," Sperling warns. "We have been involved in several other cases in China and Malaysia. I have also heard rumours about this in Thailand." Practitioners maintain that cases like these are rare and point to quality control measures they have in place to pick up inaccuracies in the data, tell-tale warning signs that something may be wrong, early on in a project. Many large agencies are investing heavily in this as part of global initiatives to uphold data quality, and the industry's reputation. Nevertheless Asia-Pacific clients who aren't careful could find themselves paying good money for bad research. MR trade body Esomar recently warned: "While the standards and quality of market research in some countries, such as Australia and Japan, match the best standards worldwide, this might not be true for all of Asia-Pacific, especially in those countries where market research has developed in the recent past." Esomar, which operates a code of professional conduct its members must adhere to, expects cases of poor research to diminish, driven by the quality control initiatives of the global chains. However, some researchers on the ground feel the issue merits greater prominence. "I have witnessed a lot of variation in quality standards in fieldwork and it seems to me that not all research agencies or buyers are paying enough attention to this," observes Wander Meijer, the managing director of the Hong Kong office for research giant, TNS. Meijer feels the root of the problem lies in the way interviewers are paid -- by results. "If an interviewer works honestly and reports every screened-out respondent when they are being paid only for successfully-completed interviews, honesty costs money." If interviewers are paid by the hour, Meijer argues, as they are in Europe, agencies will compete less on cost and more on improving services. Not everyone agrees that the piece-rate system should be abolished, arguing that in-built safeguards deter dishonest behaviour, but few dispute that a cost-driven marketplace inhibits the development of high standards. "Often, research is driven too hard, too far, too often on price," comments Steve Garton, director of media for Synovate, the global research chain housing former regional network, Asia Market Intelligence. "Quality comes at a price." Although the research industry is starting to recover after some lean years, competition remains fierce as agencies vie for research budgets set at a time when the future looked less rosy. Brand owners are faced with a tricky trade-off, seeking cost effective, yet reliable results. One buyer for a mid-sized Chinese company, who preferred not to be named, thinks local providers will feel the squeeze. "I know some local companies are making great effort. Yet it also costs much. Companies like us certainly want cheaper service. It is a dilemma really. Only big research brands can survive in such conditions." Local agencies argue that their market knowledge and cultural understanding gives them an edge over the large research groups. Foreign-owned research companies, conversely, feel local firms mainly compete on price. The Chinese Marketing Research Association (CMRA) is sufficiently concerned about agencies bending the rules to remain competitive to consider setting minimum rates for different research techniques, in conjunction with the Chinese Government, to discourage fraud. "If the price is too low, it must happen," says secretary-general Jason Ho. The CMRA is also cracking down on so-called professional respondents, regular participants in focus groups and hall tests willing to lie about their eligibility in order to collect the incentives agencies pay for taking part, compiling a register of known offenders. However, with only three employees, its ability to police the emerging Chinese market, already the second largest in the region after Japan, is limited. Many research buyers, particularly multinationals, run their own quality checks, sending in-house researchers to supervise interviewer briefings and listen in to interviews. Annette Nazaroff, consumer insights director for MindShare Asia-Pacific, hires a separate research company to audit another's work, a policy she feels is good practice anywhere in the world. "Often, research is time-specific," she says. "We can't afford to go back into the field for another month." Nazaroff feels that if a company is paying a lot of money on research, it is worth also buying some insurance to ensure reliable results. Fieldwork in some Asian countries can be as costly as it is in Europe, and companies seeking trustworthy information in these markets must be prepared to open their wallets. Cheaper research doesn't necessarily lead to a decline in standards, according to Alastair Gordon, managing director R&D and brand health, ACNielsen Asia-Pacific. "Better procedures and better processes that are more efficient usually lead to better quality," he argues. Agencies able to offer a bespoke service catering for the specific needs and issues of individual brand-owners are also more likely to have their prices regarded in a good light. "Even in the boom years, clients would never pay what we wanted," Gordon says. "It's always been a competitive industry." The biggest headache for agencies is keeping tabs on the paper trail associated with traditional face-to-face interviewing, sometimes the only option in developing markets. Agencies feel many of the concerns over the reliability of fieldwork will disappear as more surveys are carried out using more sophisticated methods. It is much easier monitoring phone interviews and easier still to keep tabs on surveys conducted with the aid of computers, whether they are on the street, on the telephone or via the web. Advertisers buying research in Asia pay much more attention to how well interviews are carried out than their counterparts in more developed markets. Agencies feel, however, that the main reason quality sometimes falls short is not down to inadequate checks on interviewers, but because they are not given enough time to ponder what the results mean. "The very short gap between commissioning research and providing the report robs us of the most critical element -- time," says Synovate's Garton. Researchers feel they are squeezed at the very point when they can add most value to a research project. The situation is exacerbated in Asia by the current talent shortage, a problem that will only be resolved by agency investment in training. Advertisers spend over US$2 billion each year to find out what people across Asia-Pacific think about their brands. The pressure on research agencies to deliver more insightful research, faster and cheaper isn't going to go away. If they can do this without compromising the integrity of what they sell, the industry's future is assured.