How do we know the Chamber of Commerce poll on charter schools is legit?

The Nashville Chamber of Commerce has made a habit of introducing its yearly "report cards" on Metro public schools with terms like "bold" to the point that it means very little: when every report card is bold, then no report cards are bold. Nonetheless, the special interest group whose main purpose seems to be to lobby and leverage policy favorable to it from Metro agencies trotted out another report card with rather interesting "poll" numbers:

Armed with a poll taken in May, chamber officials insist Nashvillians want “school choice” in the form of charters. A survey of Nashvillians, featured in the report, found that 43 percent of respondents believe the city should have as many charter schools as possible, and 39 percent say it should operate a limited number. Eleven percent believe charters are not necessary here.

Any marketer know that you can call just about anything a poll, because there are no rules governing the Chamber when it gathers its "findings". Too bad the journos at the Tennessean do not seem to heed it enough to probe into methodology.

We can assume that the "poll" was paid for by the Chamber, which makes it suspect to begin with. Money changes everything, including claims to objectivity, which brings me to my next point. I searched high and low for information on a polling agency that the Chamber could have contracted to conduct the study, but I could not find that information. Assuming that they hired a pro polling agency (I give them the benefit of the doubt that they did not just phone bank a bunch of their own volunteers asking leading, push-polling questions), I would like to know if that agency acted independently to cull these numbers in the same way an independent auditor would turn over objective balance sheets and neutral assessments. I would also like to see sample questions to test their objectivity. It is only fair. The Nashville Chamber of Commerce has not been bashful about questioning the methodology of more objective media polls at odds with their priorities.

Otherwise, I have serious doubts that nearly half of Nashville would like as many charter schools as possible. If the numbers are accurate and based on operationalized data gathering, then I can only feel sorry for a city that seems bent on going from foundering public education to mediocre privatized education that hands over tax dollars to entrepreneurs who have little or no obligation to taxpayers.


UPDATE:  Local reporter J.R. Lind tracked down the info that I could not find initially on the MNPS website. Appendix A has the questions asked and the results, but J.R. underscored the following paragraph in response to this post:

The following graphs represent results from a telephone survey commissioned by the Nashville Area Chamber of Commerce. The survey was designed, written and analyzed by McNeely Pigott & Fox Public Relations in Nashville. The Parker Consulting Group of Birmingham, Ala., randomly surveyed 545 Davidson County registered voters May 8-9 and 13-14, 2012. The survey has a margin of error of approximately plus or minus 4.2 percent for the total sample.

Essentially, a special interest lobby group commissioned a survey designed, written and analyzed by a public relations firm (which itself has ties to local election and policy campaigns, most notably those of our pro-charter Mayor) and conducted by a marketing consulting business. The matter of whether the framing of the questions as well as of the answers offered to subjects is a legitimate one that I do not want to pass over lightly (for instance, it can be leading to frame the "unlimited charters" question by mentioning state legislative action), but more important for me is the fact that at no point along the process is the research conducted by neutral, independent firms that would be more exhaustive about objectivity and fact gathering.

The Birmingham consulting firm markets their phone surveys to get "accurate information" from their client's "target audience" and to "manage to specific quotas" in the target audience. What the appendix does not say is how the target audience was selected. It also does not delineate controls in place to keep callers from phrasing questions in leading ways or straying from a strict protocol of asking the same question in the same way to each subject in the target audience.

In my opinion a more neutral and objective survey of attitudes about charter schools should be formulated and conducted by a research (not marketing) firm without skin the game or financial connections to a local special interest group that has invested years and money clips on getting answers it expects. If the results are legitimate and objectively gained, I would like to see more qualitative data on what the 43% who support unlimited charters schools believe they are supporting and why they support such a market model for public education when running government like a business has not consistently worked.