Piloting LQAS in Somaliland
By Tom Oguta, Grainne Moloney and Louise Masese
Tom Oguta has been working with FAO/FSAU in the Nutrition Surveillance Project in Somalia as a Nutrition Project Officer for the last two years. Previously, he worked as a Research Officer at Kenya Industrial Research & Development Institute and has a specific interest in sampling approaches in nutrition assessments.
Grainne Moloney is the Nutrition Project Manager of the FAO/FSAU Nutrition Surveillance Project, where she has worked for the last 18 months. Prior to FSAU, Grainne worked with UNICEF in Darfur, also on nutritional surveillance, and previously with Oxfam and Action Against Hunger UK in many different countries and contexts.
Louise Masese works with FSAU as a Nutrition Project Officer, planning and coordinating nutrition assessments. Prior to FSAU, Louise worked with a consultancy firm as a survey specialist, with WFP's Kenya Emergency Drought and Flood Operation and with UNICEF Kenya.
The authors would like to express sincere thanks to the FSAU nutrition team in the implementation and write up of this report, to the partners UNICEF and the Ministry of Health Somaliland, to the local authorities of Somaliland for all the logistical support in the implementation of the study and to the community of the IDP camps, who participated in this study.
Measuring weight and
height of a child.
This article presents a pilot study1 conducted in Somaliland by the UN Food and Agricultural Organisation Food Security Analysis Unit, (FAO/FSAU). It uses the Lot Quality Assurance Sampling (LQAS) method to assess the nutritional situation and compares the results to a 30x30 cluster survey conducted simultaneously in the same sample population.
The North West Region of Somalia, whose borders follow those of the former British Somaliland Protectorate, declared itself independent (Republic of Somaliland) in May 1991 (see Map 1). However, the declaration of independence is not recognised in other parts of Somalia or internationally. The region has, nonetheless, managed to avoid the protracted conflict and violence that has afflicted much of southern Somalia. Hargeisa town is the biggest urban centre in the region and is the capital town of Somaliland. Hargeisa is the concentration for public administration private sector, and hosts large numbers of persons from the international aid community. Due to its dynamic market and labour opportunities, it is also the destination for many refugees, returnees and internally displaced persons (IDPs). Hargeisa currently hosts more than 80,000 IDPs.
Since 1997, large numbers of UNHCR's 'official' returnees to Somaliland from camps in Ethiopia have selected Hargeisa as their chosen destination. Several returnee settlement areas have sprung up on the outskirts of the city and have grown considerably since UNHCR began its voluntary repatriation programmes in 1997. Many returnees have been settled in permanent settlement areas, namely Sheikh Nur, Mohammed Mooge, Aw Aden and Ayaha. Most others remain in temporary settlements in three different areas (State House, Stadium, A and Daami) and many more are unaccounted for, scattered in different sites within the municipality. Another poor settlement called Sheedaha is also emerging in the northern part of Hargeisa town, mainly inhabited by the urban poor who are unable to pay for rents for better housing in town. These comprise pastoral dropouts who lost their animals in previous droughts and are seeking alternative livelihoods, and those who have currently been displaced from the renewed conflicts in Mogadishu.
Nutritional status
The returnees and IDPs have been identified as nutritionally vulnerable over the past few years, reflected in a series of nutrition surveys that have found rates of global acute malnutrition (GAM) over 10%. These high rates result from inconsistent and limited income generating opportunities and lack of basic services such as sanitation, protected water and health care. Routine assessments are conducted on this population to monitor the nutrition situation and inform appropriate response. In September 2007, the Food and Agricultural Organisation (FAO) Food Security Analysis Unit (FSAU) with UNICEF, Ministry of Health and partners, conducted one 30x30 cluster nutrition survey and two exhaustive surveys in the three most concentrated and protracted IDP populations in Somaliland - Hargeisa, Berbera and Burao. The opportunity was also taken to trial an alternative sampling methodology, Lot Quality Assurance Sampling (LQAS), to assess its sensitivity in determining the nutrition situation.
LQAS and prevalence of malnutrition
LQAS is a method of sampling derived from the manufacturing industry and, over the last two decades, has been also applied to the health sector (see Box). More recently, adaptation of LQAS principles has been explored in relation to estimating the prevalence of GAM. The sampling method traditionally used to assess the prevalence of acute malnutrition in emergencies is a 30x30 cluster survey. This method provides statistically reliable results if implemented correctly but with a sample size requirement of 900, it can be time-consuming and expensive to carry out. FANTA (Food and Nutrition Technical Association2) has been exploring the use of LQAS as a rapid and cost-effective alternative for assessment of the prevalence of acute malnutrition. A study by FANTA, Catholic Relief Services (CRS), and Ohio State University (OSU)3, field-tested the use of the LQAS designs in an emergency setting in Ethiopia4. The study concluded that LQAS designs provide statistically appropriate alternatives to the more time-consuming 30x30 cluster survey, though the variance is larger, resulting in wider confidence interval round the point prevalence, and additional field testing was recommended.
FSAU Pilot
In September 2007, FSAU undertook to pilot and field-test the LQAS methodology among the IDPs in Hargeisa, in order to compare the findings with a 30x30 assessment and explore its potential application in the nutrition surveillance system in Somalia.
There are two sampling options for the estimation of the prevalence of acute malnutrition using LQAS; a 33x6 cluster design, where 33 clusters are selected with 6 children per cluster, and a 67x3 cluster designs, where 67 clusters are selected each with 3 children. Although both methods provide similar precision in relation to estimation of malnutrition, the latter has shown a higher precision for household level data. However, as 67x3 clusters requires more travel time, FSAU opted to trial the 33x6 approach.
Lot Quality Assurance Sampling (LQAS)
LQAS is a method of sampling derived in the
1920s for assessing quality of lots (or batches) of
products in the manufacturing industry. By the
1980s, the LQAS sampling concepts were recognised
as having universal applications and the
approach is now being used all over the world to
assess coverage of maternal and child health,
family planning and HIV/AIDS programmes, quality
of health workers performance and disease
prevalence. It is based on the principle that
inspection of a small, representative sample of a
'lot' will, with high probability, allow for the
valid rejection of the entire lot, should the
number of defective goods in that sample exceed
a predetermined allowable number.
A global review (covering a total of 805 LQAS surveys) of the use of LQAS surveys to assess aspects of health care including service delivery, health behaviour and disease burden was carried out by the World Hleath Organisation (WHO) in 2006. LQAS surveys were found to be a practical field method increasingly applied in the assessment of preventative and curative health services and for measuring variation in behaviour change when collected recurrently at multiple points in time. Most LQAS surveys have been used to assess risk factors for HIV/AIDS and sexually transmitted infections, although substantial numbers have also been conducted to assess immunisation coverage, growth and nutrition, and post-disaster health status of communities.
Methodology
A 33x6 cross-sectional LQAS assessment was conducted alongside a standard 30x30 cluster assessment, amongst the protractedly displaced populations concentrated in seven sites5 in Hargeisa town of Somaliland. Both studies used the same sample frame and were conducted by two separate teams concurrently.
For both studies, a two-stage cluster sampling methodology was used to select the clusters and the households. In the case of the LQAS, 33 clusters were drawn from the sampling frame, while 30 clusters were selected for the 30x30 cluster surveys. The clusters were randomly selected using the Nutrisurvey6 software. The recommended SMART (Standardised Monitoring and Assessment of Relief and Transitions) method7 was used for the second stage sampling of households and children. The same tools were also used for both studies, with quantitative data collected through a standard household questionnaire for nutrition assessment. This included data on child anthropometry, morbidity, vitamin A supplementation, measles and polio immunisation coverage, dietary diversity, and access to water and sanitation. Qualitative data were collected through focus group discussions and key informant interviews by an interagency team comprised of assessment supervisors and coordinators, to provide further understanding of possible factors influencing nutritional status.
For the LQAS study, as all eligible children in a sampled household were assessed, this resulted in a total of 204 children. For the 30x30 surveys, a total of 905 children were sampled. Two households overlapped between the two studies. As only 198 children were required for the LQAS analysis, the six extra children were randomly eliminated at the analysis stage using a table of random numbers. For both studies, household and child data were entered, processed (including cleaning) and analysed using EPI68 and Nutrisurvey software.
| Table 1: Decision Rules for 33X6 LQAS design (n=198) for various GAM thresholds |
| Decision Rule - Number of children identified as acutely malnourished in a 33x6 LQAS study |
Outcome |
| ? 13 |
GAM is < 10% |
| > 13 but < 23 |
GAM is ? 10% |
| > 23 but < 33 |
GAM is ? 15% |
| > 33 |
GAM is ? 20% |
Thresholds and Decision Rule (DR) for LQAS
Measuring MUAC
LQAS is a hypothesis test to determine whether an outcome is ? or < a defined threshold. LQAS uses two thresholds, an upper and a lower threshold, to define the alpha and beta (? and ?) errors (tolerable statistical error)9. Therefore, to design an LQAS sampling plan, the threshold of interest for an indicator (e.g. GAM prevalence), and tolerable statistical error (? and ?) are defined in advance. The upper threshold is the threshold at which the area is at risk (e.g. ?10% for Hargeisa IDPs). The alpha error is the probability of incorrectly classifying an area as not being at risk when the true GAM prevalence is ?the threshold of interest. For the purposes of this pilot study, the lower threshold is the GAM prevalence at which an area would not be considered a priority for an intervention (e.g. ? 5%) and the beta error is the probability of incorrectly classifying an area as being at risk when the GAM prevalence is < the threshold of interest.
The null hypothesis assumes the GAM prevalence is ? the upper threshold. To classify GAM prevalence as ? or < the upper threshold, the number of children with GAM is counted and then compared against a Decision Rule (DR) determined using binomial probabilities10. GAM prevalence is judged as ? the upper threshold if the number of children with GAM in the sample is > than the DR. Similarly, GAM prevalence is judged as < the upper threshold if the number of children with GAM is ? than the DR. This is illustrated in Table 1 based on a 33x6 LQAS study. Here, if more than 13 children but less than 24 children are identified as acutely malnourished, then the global acute malnutrition rate can be estimated to be 10-15%.
| Table 2: Comparative Analysis of findings using 30x30 standard cluster and 33x6 LQAS methodologies |
| Indicator |
Prevalence |
95% CI |
CI Width |
Standard Error |
Design Effect |
| |
30x30 |
33x6 |
30x30 |
33x6 |
30x30 |
33x6 |
30x30 |
33x6 |
30x30 |
33x6 |
| Global acute malnutrition (GAM) (WHZ) |
10.3 |
9.6 |
8.4 - 12.2 |
6.1 - 13.1 |
± 1.9 |
± 3.5 |
0.06 |
0.08 |
0.91 |
0.72 |
| Severe acute malnutrition (SAM) (WHZ) |
1.1 |
1.0 |
0.5 - 1.7 |
0 - 2.4 |
±0.6 |
±1.4 |
0.003 |
0.007 |
0.70 |
0.98 |
| Stunting (HAZ) |
19.2 |
19.2 |
16.3 - 22.2 |
12.7 - 25.7 |
± 3.0 |
± 6.5 |
0.09 |
0.10 |
1.30 |
1.42 |
| Underweight (WAZ) |
18.0 |
20.7 |
14.8 - 21.3 |
15.0 - 26.4 |
± 3.3 |
± 5.7 |
0.06 |
0.08 |
1.68 |
1.02 |
| Reported diarrhoea |
15.3 |
13.6 |
11.7 - 19.9 |
7.9 - 19.4 |
± 1.8 |
± 5.8 |
2.07 |
2.94 |
2.91 |
1.46 |
| Reported acute respiratory tract infection |
15.4 |
19.2 |
11.3 - 19.4 |
11.9 - 26.5 |
± 2.3 |
± 6.3 |
2.08 |
3.71 |
3.00 |
1.74 |
| Reported febrile illness |
3.0 |
2.5 |
1.2 - 4.7 |
0.5 - 4.6 |
± 1.4 |
± 2.1 |
0.90 |
1.06 |
2.53 |
0.90 |
| Suspected measles |
2.2 |
1.6 |
0.82 - 3.64 |
0.0 - 3.9 |
± 1.3 |
± 2.3 |
0.72 |
1.17 |
2.03 |
1.65 |
| Measles immunisation |
58.3 |
57.7 |
52.3 - 64.3 |
43.6 - 71.8 |
± 3.4 |
± 14.1 |
3.08 |
7.20 |
3.33 |
4.01 |
| Vitamin A supplementation |
60.9 |
43.4 |
53.5 - 68.2 |
33.4 - 53.5 |
± 3.4 |
± 10.1 |
3.74 |
5.13 |
5.33 |
2.12 |
| Polio immunisation |
89.4 |
93.9 |
86.4 - 92.4 |
88.7 - 99.2 |
± 2.2 |
± 5.3 |
1.53 |
2.70 |
2.23 |
2.53 |
| Dietary diversity (>3 food groups) |
80.9 |
90.6 |
76.5 - 85.3 |
84.1 - 97.0 |
± 3.8 |
± 6.5 |
2.23 |
3.30 |
1.63 |
1.35 |
| Access to safe water |
84.7 |
97.2 |
75.9 - 93.5 |
94.2 - 100 |
± 8.8 |
± 3.0 |
4.50 |
1.55 |
7.82 |
0.94 |
| Access to latrine |
68.4 |
74.5 |
58.2 - 78.6 |
65.1 - 82.5 |
± 10.2 |
± 9.4 |
5.18 |
4.73 |
6.25 |
1.25 |
WHZ: Weight for height z score
HAZ: Height for age z score
WAZ: Weight for age z score a
Results and Discussion
Overall, the 33x6 LQAS design produced more or less similar results compared to the conventional 30x30 design for the child data (malnutrition, morbidity and health programmes coverage) but less correlation for the household data (household dietary diversity, access to water and access to sanitation facility). However, as expected given the smaller sample size of the LQAS design (198 children), the standard error is larger, and consequently the confidence intervals are wider for the LQAS results. The design effects were generally lower for the LQAS design (See Table 2).
Analyses of the findings from the two assessment designs provide similar estimates of acute malnutrition. An acute malnutrition rate (WHZ<- 2 and/or oedema) of 9.6% (CI: 6.1 - 13.1) was reported using the LQAS (33x6) design. Comparable results (confidence intervals overlapping) were reported from the conventional 30x30 design with acute malnutrition rates (WHZ<-2 and/or oedema) of 10.3% (CI: 8.4 - 12.2). For hypothesis testing against a threshold of below or above GAM of 10%, 19 children were found to be acutely malnourished, indicating levels above 10%11 and <15% according to the DR shown in Table 1.
Conclusions
The LQAS approach required fourteen less person days and cost approximately 60% less ($5,600 compared to $13,700) than the 30x30 approach. A further benefit was to the staff who reported being less tired and more motivated.
Given these findings, there would appear to be a role for LQAS in the nutrition surveillance system for Somalia - especially for filling information gaps during the seasonal assessments and for areas with limited accessibility. In addition, given the chronically high rates of acute malnutrition reported in Somalia, this method could be used to identify hot spots and help prioritise interventions where the nutrition situation has deteriorated significantly. There is a considerable amount of baseline nutritional information in Somalia which would allow for this type of comparison.
However, for interpretation purposes, more emphasis will be needed on adapting the decision rule approach to provide a range estimate rather than an absolute prevalence estimate, given the small sample and the wide confidence intervals. This may, in turn, limit the use for monitoring the situation over time except when a significant change in the nutrition situation has occurred.
FSAU is planning to conduct further studies using the LQAS approach in less secure and accessible areas in Southern Somalia in 2008. As with this study, parallel standard surveys will be conducted to allow for comparability of the results. The results of these studies will be reported in the monthly Nutrition Updates.
For more information, please contact: Tom Joseph Oguta or Grainne Moloney, P.O. Box 1230- 00621, Nairobi, Kenya. Tel: 254-20-3745734/1299 or 0722392499 Fax: 254-20-3740598 email: tom.oguta@fsau.or.ke or grainne.moloney@fsau.or.ke
1The results of this study have been published in the FAO/FSAU monthly Nutrition Update, August 2007.
2More information on LQAS is available on the FANTA website: www.fantaproject.org
3Deitchler et al (2007). A Field Test of Three LQAS Designs to Assess the Prevalence of Acute Malnutrition. International Journal of Epidemiology. May 2007.
4Field testing LQAS to assess acute malnutrition prevalence. Summary of published research. Field Exchange 31. September 2007, p4
5Ayaha, Aw Aden, Sheikh Nur, Daami, Mohamed Mooge, Stadium, and State House.
6http://www.nutrisurvey.de/
7The method involves a modification of the standard Expanded Programme on Immunisation (EPI) method to reduce the bias in sampling the households in the centre of the settlement
8http://www.cdc.gov/epiinfo/Epi6/ei6.htm
9Megan D et al (2007). A field test of three LQAS designs to assess the prevalence of acute malnutrition. International Journal of Epidemiology. May 2007, pp 2-5.
10Valadez J, Weiss B, Leburg C and Davis R. (2003). Assessing Community Health Programs: Using LQAS for baseline surveys and regular monitoring. London: Teachingaids at low cost.
11Only 13 acutely malnourished children are required to hypothesize GAM levels of ?10%.
Taken from Field Exchange Issue 33, June 2008
http://fex.ennonline.net/33/piloting.aspx