pisa 2015 not comparable methodology east asia

Falls in Asian high-flyers' math scores could simply be down to switch to computer tests

The man running the world’s most influential education study has admitted that seemingly dramatic changes in performance for top-ranked countries shown by its “comparable data” could, in fact, be explained by changes to the way its tests are delivered.
According to the latest Programme for International Student Assessment (Pisa) study from 2015, the global top-ranked performers in maths have all seen falls in the percentage of their pupils achieving high test scores in the subject over the previous three years.
That apparent decline in the ability of East Asian maths superpowers to stretch the brightest could have wider implications. Schools in the US and the UK have invested heavily in emulating the Asian maths “mastery” approach.
But now Andreas Schleicher, the official in charge of Pisa, has said that this fall may not be due to a drop in the performance of these Asian powerhouses. He said he was looking into whether the decline could be explained by the fact that Pisa used computers for the main tests for the first time in 2015.
In other words, data that is clearly presented as “comparable” in the study may not be comparable at all.
The admission has led critics to question the whole reliability of Pisa and to call for the Organisation for Economic Cooperation and Development (OECD), which runs the study, to be more open about its limitations.
According to the 2015 Pisa study, all six of the top-ranked systems in maths with comparable data saw falls in the percentages of their pupils with top levels of attainment in the subject, compared with the previous 2012 study.

South Korea, Taiwan and Hong Kong experienced respective declines of 10, 9 and 7 percentage points in the proportions of students with the ability to select and evaluate appropriate strategies for complex problems. Singapore, Japan and Macao also saw drops.

China, the other top-ranked performer in maths, had no comparable data as it entered only as Shanghai in 2012.
When asked why he thought these education superpowers had all dropped in performance, Mr Schleicher admitted they might not have done so at all. The OECD education director suggested that the move to computer-based tests might be the reason.
“Further analysis is needed to establish the causes of decline in the share of top performers in some of the highest-performing countries,” he said.
He said although the study had ensured that, “on average”, pupils taking paper- and computer-based tests scored the same, that might not be true for some groups of high-performing pupils.
“It remains possible that a particular group of students – such as students scoring [high marks] in mathematics on paper in Korea and Hong Kong – found it more difficult than [students with the same marks] in the remaining countries to perform at the same level on the computer-delivered tasks,” he said.
“Such country-by-mode differences require further investigation – to understand whether they reflect differences in computer familiarity, or different effort put into a paper test compared to a computer test.”
But there is no mention of that possibility alongside the data in the report showing the change in the percentage of top-performing students between 2012 and 2015. The report clearly says the data is “comparable”.
The possibility that the change to computer tests could have made a general difference is covered elsewhere in the study, but then largely discounted.

Pasi Sahlberg, an expert in global education reform, said that Mr Schleicher’s admission could have wider implications.

“It raises new questions about the reliability of the [Pisa] test itself,” he said. “Students’ measured literacies in reading, mathematics and science should not depend on how they are measured, if the scope of testing remains the same.”

iq sociological factors china

In “Race, IQ, and Wealth,” I examined the pattern of IQ scores for various European peoples as presented by Richard Lynn and Tatu Vanhanen in IQ and the Wealth of Nations and noted the considerable evidence for a large socio-economic influence. In nearly all cases, impoverished, rural populations seemed to exhibit far lower IQ scores than affluent, urban ones, even when the populations compared are genetically indistinguishable. Furthermore, these lower IQs often rise rapidly once conditions improve, in what might be called a “Super-Flynn Effect.”  However, this strong relationship between wealth and nominal IQ seems to disappear when we examine East Asian populations. A few decades ago, China, Taiwan, Hong Kong, Singapore, South Korea, and even Japan had extremely low per capita GDPs relative to those of America or Europe, yet almost all their tested IQs were around 100 or higher, comparable to those of the wealthiest and most advanced European-derived nations. In many cases, their incomes and standards of living were far below those of the impoverished nations of Southern and Eastern Europe, yet they showed no signs of the substantially depressed performance generally found in these latter countries, whose IQs were usually in the 88–94 range. This can be seen in the table below.  For consistency, all these results are drawn directly from Lynn/Vanhanen, and include their Flynn and other IQ adjustments up and down, several of which seemed rather large and arbitrary, with the GDP obtained from the World Bank, adjusted for Purchasing Power Parity (PPP 2005$) unless indicated by an asterisk. Much of this economic data is somewhat uncertain and should be used only for rough comparative purposes. A wide range of additional IQ results from these same countries are found in their 2006 sequel, but these lack testing-date information, making it impossible to compare with income levels or discern historical trends, and they anyway seem to fall into the same range.     This clear pattern of East Asian IQs remaining almost unaffected by depressed socio-economic conditions had also occurred when such ethnic populations lived as small minority groups in America. Whereas in the early decades of the 20th century schoolchildren whose families had immigrated from Southern and Eastern Europe tended to have very low tested IQs, often in the 80–85 range, most studies of that era showed that children from Chinese-American and Japanese-American immigrant backgrounds had IQs similar or even superior to the white mainstream population, despite their much lower socio-economic backgrounds.  One possible explanation of this striking result might be that these East Asian test results actually were artificially depressed due to relative deprivation and that once this condition was alleviated, Asian scores would rapidly rise by the same amounts as had those of various European-origin groups in different periods, perhaps 10–15 points. But this would imply that the fully-adjusted mean IQ scores of East Asians might approach the 120 range, and this seems unlikely, since affluent, well-educated present-day Asian nations such as Japan or South Korea show no evidence of mean IQs so high.  Indeed, the most obvious aspect of the East Asian IQs shown in the table below is that they bear almost no relationship to the wealth of the countries at the time the testing was performed. For example, Japan in 1951 was desperately poor, and its real per capita GDP rose tenfold during the 40 years that followed, but its IQ rose just a couple of points. Similar huge rises in income without significant rises in IQ occurred in South Korea, Taiwan, and other countries. The 2006 sequel by Lynn and Vanhanen provides numerous additional IQ reports from East Asian countries, but they all continue to fall into this same general range of scores. Furthermore, Asian-Americans living in the United States these days are generally affluent, but although they perform very well in school, their tested IQs do not have a mean anywhere near 120.  The most plausible inference from these decades of accumulated data is that the IQs of East Asian peoples tend to be more robust and insulated against the negative impact of cultural or economic deprivation than those of European groups or various others—a truly remarkable finding. This might be due to cultural factors of some type, or perhaps certain aspects of East Asian spoken or written languages. But a fascinating possibility is that this IQ robustness may have a substantially genetic component.  This would be somewhat similar to various physiological findings in recent years. For example, health studies in America have repeatedly shown that individuals of East Asian ancestry tend to have significantly longer life expectancy and lower rates of illness than most other American ethnic groups, and this effect seems independent of other environmental or dietary inputs and persists even after controlling for socio-economic factors. Over one hundred years ago, The Changing Chinese by A.E. Ross, one of America’s greatest early sociologists, provided copious anecdotal evidence indicating greater Chinese resistance to illness and injury and perhaps even an ability to survive on more meager food rations. Certainly these sorts of traits might be expected to have undergone strong selection in a country such as China, whose huge population had lived many centuries at the absolute Malthusian edge of starvation.  With regard to mental traits, decades of testing have established that the intelligence subcomponents of East Asians and Europeans are somewhat different in structure, with East Asians being relatively stronger in spatial ability and Europeans stronger in verbal ability. Since these differences are also found in East Asians raised and acculturated in America and other Western countries, they seem to have a large genetic component. Although this particular result was less well established at the time, the general notion that different groups might have differing relative strengths in particular abilities was the centerpiece of Howard Gardner’s famous “Theory of Multiple Intelligences,” publicized in his 1985 book Frames of Mind, which has received widespread attention in media and educational circles over the last couple of decades.  Although the precise genetic basis of the differing East Asian and European skews in mental ability has not been determined, some corresponding physical traits have already been localized in recent genetic studies, notably skin color. Both Northeast Asians and Northern Europeans tend to have relatively pale skin, presumably due to the evolutionary pressure they experienced to synthesize maximal amounts of Vitamin D under weak sunlight during the thousands of years they lived in northern latitudes. But in the last decade, we have discovered that the particular genetic mechanisms that they evolved to block melanin production and produce lighter skin are dissimilar, having developed via entirely different mutational pathways.  To the extent that East Asian IQs are indeed far less vulnerable to negative socio-economic factors than those of other racial groups, recognizing this fact might make it far easier for us to admit the important role that such environmental influences might play in determining the nominal IQs of other populations.

china catches up the u.s. in science

TOKYO China now ranks as the most influential country in four of eight core scientific fields, matching the U.S., according to the Japan Science and Technology Agency.  And with U.S. President Donald Trump planning a major spending cut for the sciences, China could well take the sole lead.  Dipping into the global database of scientific theses, the agency took the top 10% of the most-referenced studies in each field and determined the number of authors who were affiliated with the U.S., the U.K., Germany, France, China or Japan. China ranked first in computer science, mathematics, materials science and engineering. The U.S. led the way in physics, environmental and earth sciences, basic life science and clinical medicine.  China's progress was especially pronounced in computer science. While the country accounted for only 3% of the most-referenced studies in 2000, the figure had surged to 21% by 2015. It has also had the fastest supercomputer in the world since 2013, and the two fastest in 2016.  The country is also rapidly catching up in physics, a field long dominated by the U.S. China is spending more than $6 billion to build the world's largest particle accelerator, which could put it at the forefront of particle physics.  The country's advances were made on the back of heavy government spending and an extensive campaign to attract talent. China's public and private spending on research was double Japan's in 2014, and is fast approaching the U.S. tally of $460 billion. The country is making efforts to bring home Chinese researchers who trained abroad, and to connect with overseas talent through overseas study programs and temporary placements.  "I was not expecting China to overtake the U.S. in many fields," said Yuko Ito at the Japan Science and Technology Agency.  Despite winning Nobel Prizes for three straight years, Japan came in at fifth or sixth place in many fields. Even in chemistry, where the country has excelled, Japan ranked fifth. Though 17 Japanese have won Nobel awards this century, most of their main studies were done over 30 years ago

grey matter asians more bigger brains

Share

Researchers from China may have just found the key to a mystery that has stumped scientists for decades: Why is the average Asian brain significantly larger compared to the average European or African?
According to a new study by the Chinese Academy of Sciences’ Kunming Institute of Zoology, the answer may, in fact, lie with Darwinian selection in East Asian populations, reported South China Morning Post.
The study, published in the journal Human Genetics late last month, posits that genetic mutations have led to bigger brains in the group due to natural selection. Such preference was notably missing in Europe or Africa.
The discovery was first put forth by a team of American researchers three decades ago after studying more than 20,000 present-day human skulls from different parts of the world in what was considered the largest survey of brain sizes.
Findings revealed that the East Asians’ average cranial capacity (the volume of the interior of the cranium) was 1,415 cubic centimeters, which was bigger than the Europeans’ which was averaged at 1,362 and 1,268 for Africans.
Other similar studies that followed also supported such findings but none could explain why.
According to the Chinese research team led by Professor Su Bing, a gene called CASC5, one of the eight genes that regulate the human brain size, may hold the answer to the puzzle.
The gene is unique since its genetic mutations in humans are relatively new, and was only triggered after our ancestors left Africa around 50,000 to 100,000 years ago.
After isolating and comparing CASC5 mutations in different groups, the researchers found a “high frequency” of four mutations that increased the brain size among East Asians. Compared with Europe or Africa, mutations (growth) are much rarer.
“At the population level, our results suggest a selection of CASC5 in East Asian populations, which seems to favour a larger grey matter volume of the brain,” the researchers wrote in the paper. “By contrast, no signal of selection was detected in Europeans and Africans.”
“Precisely why this occurred is not entirely clear,” they added.
The scientists, however, points out that the drivers behind such change are still unclear for the time being. While theories exist, Su admits that such would just be speculation at this stage.
“Precise answers require further studies,” he stated.
Su, however, clarified that the size difference does not account for any intellectual advantage of the Asian brain over others.
“Scientific research has found no evidence, none at all, to support the existence of intellectual difference among races,” he said.
He further pointed out that as humans evolve and the brain increases in size, it requires more energy and resources it from the rest of the body. Some scientists believe that this has resulted in, among others, a decrease in physical strength.
Further investigation to substantiate such observation, however, is required, according to Su.
“The Darwinian selection may still be going on today, but I think the brain size difference among races will eventually disappear due to the widespread genetic exchange occurring around the world today,” he said.

brain efficiency

Mental chronometry

Main article: Mental chronometry

Mental chronometry measures the elapsed time between the presentation of a sensory stimulus and the subsequent behavioral response by the participant. This reaction time (RT) is considered a measure of the speed and efficiency with which the brain processes information.^[145] Scores on most types of RT tasks tend to correlate with scores on standard IQ tests as well as with g, and no relationship has been found between RT and any other psychometric factors independent of g.^[145] The strength of the correlation with IQ varies from one RT test to another, but Hans Eysenck gives 0.40 as a typical correlation under favorable conditions.^[146] According to Jensen individual differences in RT have a substantial genetic component, and heritability is higher for performance on tests that correlate more strongly with IQ.^[147] Nisbett argues that some studies have found correlations closer to 0.2, and that the correlation is not always found.^[148]
Several studies have found differences between races in average reaction times. These studies have generally found that reaction times among black, Asian and white children follow the same pattern as IQ scores.^{[149][150][151]} Rushton & Jensen (2005) have argued that reaction time is independent of culture and that the existence of race differences in average reaction time is evidence that the cause of racial IQ gaps is partially genetic instead of entirely cultural. Responding to this argument in Intelligence and How to Get It, Nisbett has pointed to the Jensen & Whang (1993) study in which a group of Chinese Americans had longer reaction times than a group of European Americans, despite having higher IQs. Nisbett also mentions findings in Flynn (1991) and Deary (2001) suggesting that movement time (the measure of how long it takes a person to move a finger after making the decision to do so) correlates with IQ just as strongly as reaction time, and that average movement time is faster for blacks than for whites.^[152] Mackintosh (2011), p. 339 considers reaction time evidence unconvincing and points out that other cognitive tests that also correlate well with IQ show no disparity at all, for example the habituation/dishabituation test. And he points out that studies show that rhesus monkeys have shorter reaction times than American college students, suggesting that different reaction times may not tell us anything useful about intelligence.

chinese janitors outpeform children of doctors in america

The latest analysis of international math scores will have some disturbing news for Canadian professionals spending loads of cash on tutoring and enrichment for their kids: Their offspring were outmatched by the children of janitors in Shanghai.

Ever since the PISA exam scores were announced in December, parents and education experts have been fretting over Canada’s 13th-place ranking in math. But when parental education is taken into account, it turns out the children of the country’s doctors and lawyers fall even further in the rankings: They placed 22nd when compared to their similarly advantaged peers around the world.

Canadian students with parents working in the least-skilled jobs, such as cleaners and couriers, may have answered, on average, fewer questions correctly than the better-off students in their class. But when ranked against their global peers, they did much better – placing 10th. (One caveat: The sample size of students by category varied between countries, sometimes significantly – Leichtenstein, for instance, recorded a very small number of students from this group, so wasn’t counted.)

The good news: Canada has one of the most equal-opportunity education systems in the world, according to the OECD study.

“We do a very good job, and put a lot of energy, into being average,” says Miles Corak, an economics professor at the University of Ottawa, who studies equality. “This is good because in not letting the least advantaged kids – in terms of family resources – fall behind, we have an overall higher score, and frankly in the long run, a more inclusive society.”

At the same time, Corak observes, “average is increasingly not good enough.”

The 2012 rankings of the PISA exams – which tests 15-year-olds in 64 countries in math and reading – raised alarms in Canada because students had continued a nearly decade-long drop in math scores, falling out of the top 10.

The latest study shows that, generally, kids from more advantaged backgrounds outperformed their less well-off counterparts, especially in math. But global comparisons were revealing. In Shanghai, which came first in international scores and where 15-year-olds outperformed all countries in every category by parental education, students from the least-skilled families were good enough to place 10th among all students with professional or managerial parents – significantly ahead of teens in countries such as Canada, Britain and the United States.

The study reveals an important story hidden within the overall rankings. For instance, while Finland outranked Germany overall in average math scores, this was because the Scandinavian country has low inequality in its education system. By contrast, while German students with parents working in manual occupations performed “very poorly,” the study found that the children of professionals in Germany were among the highest achievers in the world.

The studies concludes that the fact that “students in some countries, regardless of what their parents do for a living, outperform children of professionals in other countries shows that it is possible to provide children of factory workers the same high-quality education opportunities that children of lawyers and doctors enjoy.”’

Since the December results, there has been a lot of debate about the validity of comparisons of diverse countries, such as Canada, to more cities such as Shanghai, the financial centre of China. The fact that China’s results are divided up by city-region on the PISA scores has been controversial, as this Brown Center on Education column points out, even though the head of PISA has stated in previous years that rural results, which are not released by China, are in line with the public results.

Many education experts have pointed out that the high scores of Asian countries are capturing a “shadow education” in which the vast majority of students, even those from low-income families, participate in private tutoring in addition to regular classes. (The Shanghai results also include a smaller percentage sample of all the 15-year-olds in the city, many of whom don’t attend public schools because of passport-type licensing system for families called “hukou” that restricts access to certain municipal services.)

But experts have also noted a key difference in the learning culture of places such as Shanghai, where achievement is consider to be the result of work, and North America, where achievement has tended to be considered more based on ability.

And for all the caveats, these results should spark a discussion about Canada’s math rankings that step outside the narrow domains of classroom and curriculum. When the average janitor’s son in Shanghai outperforms Canadian students with every advantage, it’s time to take a hard look at the big-picture cultural messages our kids are getting about resilience, grit, and the achievement to be found in hard work.