Researchers, policymakers, and parents around the world are increasingly aware of the critical importance of child development in the first years of life.
Children’s cognitive skills – from reasoning to sustained attention and language – are affected by their early experiences, including physical health, interactions with adults and peers, and formal early childcare and education programmes. In order to develop programmes and interventions for all children to reach their developmental potential, it is essential to understand what is meant by child development and to agree on how it can be measured in different contexts.
Despite growing interest in child development globally, data on child development from low- and middle-income countries remain scarce. One major reason for the paucity of such data is that most internationally used measurement tools were developed in the United States or other developed countries, and these child populations are unlikely to be representative of the world’s children. These assessments therefore may not be appropriate for use in developing countries. At best, the use of western test instruments with uncertain validity may lead to poor psychometrics and inability to precisely estimate the effects of an intervention or policy on child development. At worst, it could induce researchers or policymakers to make inaccurate comparisons between children in developed countries and children in developing countries.
“Most internationally used measurement tools were created in developed countries, where child populations are unlikely to be representative of the world’s children.”
In 2009, the Zambia Early Childhood Development Project (ZECDP) was created to examine these issues. Its objective was to develop a tool that could 1) yield internationally comparable, multi-domain measures of child development; 2) be sensitive to local culture and linguistic differences; and 3) be adapted to other developing countries. After an initial review of the existing literature, a technical advisory team was formed in Lusaka, composed of members from the University of Zambia, UNICEF Zambia, the Examination Council of Zambia and the Center on the Developing Child at Harvard University. Based on the existing literature, seven fundamental domains of child development were identified for measurement: fine motor skills, language (expressive and receptive), non-verbal reasoning, information processing, executive functioning, socio-emotional development and task orientation. The final version of the assessment included western-developed assessments, heavily adapted assessments based on common tools, and several locally developed assessments. The assessment battery was administered to a nationally representative sample of more than 2,500 six-year-olds in 2010 and 2012.
During the ZECDP piloting process, the team found that many western-developed assessments were inappropriate for the Zambian context. Some had low-face validity; consider, for example, a vocabulary assessment asking children to identify an igloo or a chemistry set. Others tested skills not relevant to or appropriate for Zambian children. For example, one assessment asked children to use scissors, a tool not generally owned by households or used by young children in Zambia. Others ask children to interact with two-dimensional stimuli, such as line drawings in books. As the majority of children in the ZECDP study lived in households without children’s books, and as children often do not begin school until age 7, this type of activity was so novel that many children simply did not respond.
To address this problem, the ZECDP developed several new assessments. One is the Object-based Pattern Reasoning Assessment (OPRA), a tool to assess children’s non-verbal reasoning skills. The logic of non-verbal reasoning tests is that children are supposed to recognize patterns (generally in pictures), and then supposed to be able to deduce how a specific sequence of images would continue. The fundamental challenge with this task is that many children may be perfectly able to recognize patterns, but are not used to seeing them on two-dimensional (printed) materials. The OPRA avoids the unfamiliar two-dimensional format by presenting patterns using stones, beads, toothpicks and other items familiar to Zambian children.
To show how different the responses are, the OPRA directly translated 10 standard paper-based questions into object-based sequences capturing the same logic. For example, the paper assessment may have a pattern as follows: green circle, red square, green circle, (blank space), green circle. The corresponding object-based item would have the following pattern: bottle cap, toothpick, bottle cap, (blank space), bottle cap. Figures 1 and 2 show how different the results for these tasks look empirically. While more than one third of children perform very poorly on the two-dimensional test (>80 per cent incorrect), the same is true only for 13 per cent with object-based testing. Just 13 per cent of children would be classified as doing well (getting > 50 per cent right) on the two-dimensional test, while the same would be true for three times as many children with the OPRA tool.
While these results do not necessarily mean that OPRA tools are better than traditional tools in predicting schooling or other later life outcomes, the results presented strongly illustrate how assessments of children’s cognitive skills depend on the specific tools used. Cognitive tests based on two-dimensional images should be avoided to the extent possible, and, if used, interpreted with caution. Tools with greater contextual validity and reliability allow programme evaluators, policymakers and researchers to better understand children’s development and needs, which in turn allows for the design of more effective interventions.