Statistician, heal thyself

The Fraser Institute has been given a poor grade in math by a BC university. The Fraser Institute’s controversial Report Card on Secondary…

The Fraser Institute has been given a poor grade in math by a BC university.

The Fraser Institute’s controversial Report Card on Secondary Schools in British Columbia and Yukon, was recently reviewed by Simon Fraser University.

The results are surprising.

Based largely on standardized tests, the institute’s report focuses on evaluating students’ skills in math and literacy in Canada’s westernmost province and territory.

Schools with the highest marks in arithmetic and English, rank highest on a scale of zero through 10.

Yukon schools fared poorly, with Watson Lake scoring the lowest, at zero.

But Simon Fraser’s review suggests the institute’s math is a tad wonky.

Its numbers are unreliable, according to the recently released Cautions About Rating BC’s Schools.

“We’re told the institute’s ratings of elementary schools, as well as the report it released in April rating secondary schools, are widely discussed,” the report reads.

“Although it doesn’t happen in BC, in the US some jurisdictions use ratings like these, along with other information, to decide how much funding schools receive.”

And what if the numbers are wrong?

“What we focused on is that they don’t include any indication of how accurate, how much measurement error there is in their number that they provide for each school,” said professor and co-author of the study John Nesbit.

All reputable polling organizations show how accurate a number is by giving a frequency (19 times out of 20) and a range (1.1 +/– for example).

“Our main issue with the Fraser Institute’s report is they did not do that,” said Nesbit.

“They never provide anything like a margin of error.”

This raises serious questions about how accurately schools are ranked.

The three academics did their own calculations.

They found that a school’s grade on the report card was accurate within a range of 1.1 on the institute’s scale.

This means that each school could actually be 1.1 point higher or lower than the grade it was given, 19 times out of 20.

“That’s a pretty wide range,” said Nesbit.

“A lot of the schools, in any one area, are often well within that range.”

This is significant.

“When our children and our collective futures are at stake, we can’t afford to behave like the novice carpenter: measure once, cut twice,” reads the report.

“Once opportunities for students and schools are cut, changing that decision can be very difficult.”

It may seem far-fetched that numbers, on one report, from one study, from one independent research firm pose a threat to our future, especially in the Yukon.

The territory’s schools have only recently been included in the index.

With more than half the categories based on exam marks for courses testable by BC provincial exams, many of the territory’s own courses are excluded.

For programs like Music, Arts, Drama and experiential science at the Wood Street school, there is no measurement on the report card.

Many prominent officials in the territory’s education system outlined numerous shortcomings in the report’s evaluation of Yukon schools.

But in Vancouver’s Lower Mainland, parents are uprooting their kids to live in districts with the top-scoring schools.

“Sometimes parents are actually selling their house and moving so they can get their children into a school that’s higher in the rankings,” said Nesbit.

“But they might be doing that for no good reason because the difference could just be based on measurement error, just a statistical blip.”

It’s also important to define what the report card actually measures, said Nesbit.

It’s based on standardized tests with heavy emphasis on numeracy and literacy.

“If your main educational goals are something different than those, of course it would not be valid,” said Nesbit.

How important teachers believe the test is can also influence the results — some take weeks to prepare their classes while others do no prep at all, depending on the philosophy of the school and the parents, added Nesbit.

With such a small number of students in some of the territory’s classrooms, the numbers may also be less accurate. The smaller the school the more room there is for error.

And the results are geared towards university entrance and favour private schools, Yukon school superintendent Lee Kubica said in an earlier interview with The News.

“A public school allows all students, regardless of ability and work ethic, and all sorts of other things that go into making a good student,” said Kubica.

By overemphasizing “the average,” the report card does not fairly evaluate schools, said Yukon Teachers’ Association president Sandra Henderson in April.

“Much can be done to explain the relative comparisons,” said Henderson.

“Socioeconomics, community conditions, physical or emotional concerns among others.”

While all of these factors question the validity of the report card, there is some value to standardized testing, said Nesbit.

“(Standardized tests) are designed for particular purposes; they are designed to help the ministry monitor schools in a certain way,” he said.

Governments likely have an understanding of what limitations are attached to these tests, he added.

“But there’s a danger in taking them out of that context and using them for something they weren’t designed for. And I think that’s what we’re seeing here.”

Get local stories you won't find anywhere else right to your inbox.
Sign up here

Just Posted

Dr. Brendan Hanley, Yukon’s chief medical officer of health, speaks at a press conference in Whitehorse on March 30. Hanley announced three more COVID-19 cases in a release on Nov. 21. (Crystal Schick/Yukon News file)
Three more COVID-19 cases, new exposure notice announced

The Yukon’s Chief Medical Officer of Health, Dr. Brendan Hanley, announced three… Continue reading

Yukonomist Keith Halliday
Yukonomist: COVID-19 strikes another blow at high-school students

They don’t show up very often in COVID-19 case statistics, but they… Continue reading

The Cornerstone housing project under construction at the end of Main Street in Whitehorse on Nov. 19. Community Services Minister John Streicker said he will consult with the Yukon Contractors Association after concerns were raised in the legislature about COVID-19 isolation procedures for Outside workers at the site. (Crystal Schick/Yukon News)
Concerns raised about alternate self-isolation plans for construction

Minister Streicker said going forward, official safety plans should be shared across a worksite

The Yukon’s Chief Medical Officer of Health Dr. Brendan Hanley, pictured at a press conference in October, announced three new cases of COVID-19 on Nov. 20 as well as a new public exposure notice. (Crystal Schick/Yukon News file)
New COVID-19 cases, public exposure notice announced

The new cases have all been linked to previous cases

Chief Superintendent Scott Sheppard of the Yukon RCMP speaks to media in Whitehorse on Nov. 19, about Project MUSKRAT which has been ongoing since December 2017. Yukon RCMP have charged five Whitehorse individuals and seized $450,000 in cash along with drugs, prohibited weapons and stolen goods after acting Nov. 4 on search warrants obtained during the three-year-long investigation. (Crystal Schick/Yukon News)
Yukon RCMP seize $450,000 and stolen goods in Whitehorse drug bust

Five individuals have been arrested and released on conditions.

Beatrice Lorne was always remembered by gold rush veterans as the ‘Klondike Nightingale’. (Yukon Archives/Maggies Museum Collection)
History Hunter: Beatrice Lorne — The ‘Klondike Nightingale’

In June of 1929, 11 years after the end of the First… Continue reading

Samson Hartland is the executive director of the Yukon Chamber of Mines. The Yukon Chamber of Mines elected a new board of directors during its annual general meeting held virtually on Nov. 17. (Joel Krahn/Yukon News file)
Yukon Chamber of Mines elects new board

The Yukon Chamber of Mines elected a new board of directors during… Continue reading

The Yukon Hospital Corporation has released its annual report for 2019-20, and — unsurprisingly — hospital visitations were down. (Crystal Schick/Yukon News file)
Annual report says COVID-19 had a large impact visitation numbers at Whitehorse General

The Yukon Hospital Corporation has released its annual report for 2019-20, and… Continue reading

Whitehorse City Hall. (Joel Krahn/Yukon News file)
City hall, briefly

A look at decisions made by Whitehorse city council this week

City council was closed to public on March 23 due to gathering rules brought on by the COVID-19 pandemic. The council is now hoping there will be ways to improve access for residents to directly address council, even if it’s a virtual connection. (Crystal Schick/Yukon News file)
Solution sought to allow for more public presentations with council

Teleconference or video may provide opportunities, Roddick says

Megan Waterman, director of the Lastraw Ranch, is using remediated placer mine land in the Dawson area to raise local meat in a new initiative undertaken with the Yukon government’s agriculture branch. (Submitted)
Dawson-area farm using placer miner partnership to raise pigs on leased land

“Who in their right mind is going to do agriculture at a mining claim? But this made sense.”

Riverdale residents can learn more details of the City of Whitehorse’s plan to FireSmart a total of 24 hectares in the area of Chadburn Lake Road and south of the Hidden Lakes trail at a meeting on Nov. 26. (Ian Stewart/Yukon News file)
Meeting will focus on FireSmart plans

Riverdale residents will learn more details of the City of Whitehorse’s FireSmarting… Continue reading

The City of Whitehorse is planning to borrow $10 million to help pay for the construction of the operations building (pictured), a move that has one concillor questioning why they don’t just use reserve funds. (Crystal Schick/Yukon News)
Councillor questions borrowing plan

City of Whitehorse would borrow $10 million for operations building

Most Read