Analysts criticize use of OECD data in Harvard report giving Canada poor broadband ranking

Harvard University’s Berkman Center for Internet and Society wasn’t expecting to create a tempest in a tea pot when it released data last week showing Canada having some of the poorest broadband service in the developed world. And while some Canadian analysts are criticizing the study – particularly its use of contentious OECD data – the Harvard researchers are standing by their results. The 232-page study, commissioned by the Federal Communications Commission (FCC), regulators, found that Canada was 22nd overall out of 30 countries. Canada ranked 16th on broadband penetration, 20th on speed and capacity, and 25th on price. Japan, Sweden and South Korea headed up Harvard's rankings, while the United States placed above Canada at 13th overall. The study made headlines across Canada. “I reviewed the CBC article and a thick slice of the other Canadian coverage, including Mr. Goldberg's blog posts,” said Seth Young, a communications officer with the Berkman Centre. Young is referring to Canadian telecom analyst Mark Goldberg, who has argued that the researchers failed to properly investigate OECD data and, as a result, reproduced mistakes. Young disagrees, and refers readers to Chapter 3 of the report which he says covers international comparisons and anticipates and deals with criticisms of the OECD data, including the charge that the organization’s household data skews inhabitant data, resulting in ‘false negatives.’ “There has been too much emphasis on one particular measure—penetration per 100 inhabitants, which is only one way of measuring one facet of what one might plausibly seek to learn from a benchmarking exercise,” says the Harvard report. Speaking with Tech Media Reports, Goldberg contends that societal realities severely alter the results. Without knowing what is going on in home, for example, there is no way to assure the accuracy of the numbers, he notes. “Households buy residential broadband, individuals don’t. And the typical Canadian household is bigger than the average household size in the OECD,” says Goldberg, who recently published a report sponsored by Canada’s largest ISP’s that has a much rosier view of the situation in Canada. Golberg also makes the point that, when it comes to pricing, the OECD only looks at data from Bell Canada, Rogers Communications, and Shaw Cable. He argues the OECD methodology is sloppy. “They double count Dansk Broadband in Denmark, and have six providers on Sweden, four in Spain, four in the Slovak Republic, five in Ireland – why only three in Canada?” A fact check has proven Goldberg right. Telus Corp, Videotron, SaskTel, MTS Allstream, as well as smaller providers, are all absent from the study. But Harvard also used International Telecommunication Union (ITU) data, which correlated well, and argued that, for example, “Canada is a second quintile performer in penetration...but only a fourth quintile performer on speeds and prices. Keeping an eye out for these kinds of discrepancies allows us to identify false ‘successes’ and false ‘failures.’” “To their credit the Berkman centre found anomalies,” says Goldberg. “But they didn’t look at other critiques of the OECD data, specifically the pricing and speed data, which are seriously flawed.” Notably, since this summer Canada’s cable companies have boosted download speeds to up to 50Mpbs, which might have helped our ranking. Report may have a big policy impact The Berkman Center researchers anticipated criticism, but also clearly wanted to learn from the international experience so as to help advise the FCC as part of Amreica’s National Broadband Plan. "We went well beyond simply accepting the existing benchmarks and statistics, like those of the OECD,” project head Yochai Benkler, the Berkman Professor of Entrepreneurial Legal Studies, and faculty co-director of the Berkman Center, said during a Q&A upon the report’s release. “We undertook additional new research, our benchmarks have many more dimensions, and we correlated independent studies with existing benchmarks to evaluate the level of confidence we can have in the findings." The result isn’t just a question of determining who is higher up the ladder, but a well-reasoned conclusion as to what kind of a broadband approach works best. Benkler’s conclusions may cause some consternation to big players north of the border. “The single most surprising and important finding of this study is that there is more or less universal agreement...that a family of regulatory policies called open-access regulation has played an important role in successful performance on broadband penetration,” said Benkler. “What we found is that where open access policies were not only enacted formally, but actually pursued by an engaged regulator, we saw new competitors entering the market, usually more innovative and agile than the incumbents.” As Michael Geist, Canada Research Chair in Internet and E-Commerce Law at the University of Ottawa, has written in response to the Harvard report: “While supporters of the Canadian status quo are sure to find fault with the study (it uses OECD and Speedtest.net data after all), the report is particularly noteworthy given that it attempts to link Canadian policy with its falling rankings.”

Analysts criticize use of OECD data in Harvard report giving Canada poor broadband ranking

News | 10/21/2009 4:12 pm EDT

Unlock all the Canadian telecom, broadcasting and digital media news you need.

FREE TRIAL

SUBSCRIBE

Reuse & Permissions

Inside The Wire Report

Information

Hill Times Publishing

Get in Touch With Us