Survey Methodology and Technical Report for Sierra Leone

Introduction
In 2008 Fondation Hirondelle commissioned Graham Mytton and the local Sierra Leone agency, CMDA, to conduct a national audience measurement survey. 

The study was conducted to assist Fondation Hirondelle to know more about audiences for Cotton Tree News, and also to measure and understand better the rapidly changing media scene throughout the country. Another important objective of the study was to learn more about how health information was learned and understood and the media that were important for health information. The survey aimed to provide comprehensive information on the audience for all media, including all the many radio stations, TV channels, newspapers and the Internet.

Methodology
Overview
The universe sampled was the adult population aged 15 years and above living in Sierra Leone.
A random sampling method was used to identify respondents participating in the study. A total of 2,099 interviews was achieved in Sierra Leone, between 7th and 25th June 2008.

Survey Area
The research study was conducted in all 4 Provinces:  East, North, South and West, and in all 14 Districts. Beyond District level chiefdoms and urban areas were randomly selected.

Universe
The population size for Sierra Leone was taken to be 3,100,000 adults aged 15 years and above as per the Development Assistance Coordination Office (DACO/SLIS) June 2006  The estimate for this study was obtained by  applying a growth rate of 2.292%. as per the CIA World Fact book. This figure is confirmed also by the population figure given for the country by the authoritative source, the Population Reference Bureau.  [1]

Sampling Procedure
Sample structure
The sample was proportionate to the census distribution of the population as measured in the 2004 national census. 

The allocation of interview numbers was proportionate to the census figures. Chiefdoms and in the case of Freetown, areas, were selected at random. And the numbers interviewed in each selected area or chiefdom was related to the population distribution. In each district, the main town was selected automatically.

The table below indicates the number of interviews achieved by Province, District and Chiefdom or Area:

            

              

                     

                      

A total of 83 Chiefdoms was selected in the provinces while in Freetown, 20 areas of the city were selected. In each case, random numbers were used to select the chiefdoms or city areas. The only exception to this rule was that in each district the main town was always selected. The selected areas and chiefdoms are listed in the above tables.

From each selected sampling point in Freetown the third house on the third street in each was selected.

Selection in Chiefdoms was very similar. The starting point for each household was usually easily identifiable points such as schools, police stations/posts, religious buildings, institutions, markets and similar. At each point, interviewers were ordered to select the third household from the third street or path. Each selected village in the chiefdom, was divided into four quadrants  which was determined by the enumerator going round the town to determine the radius. for each quadrant,
The   first house and subsequent each third house was visited to start the selection of the household, this was selected on the left and right as he/she considers each direction.

In all areas, the interviewer applied the Kish Grid that was provided on every questionnaire for the selection of respondents. At each household in this process, the number of people in the household aged 15 and over was listed; starting with the oldest member down to the youngest.

Interviewer briefings
The training sessions in all regions brought together both the enumerators or interviewers and their supervisors with a view to familiarizing them with the questionnaire. During this process, interviewers and supervisors were taken through the questionnaire in a detailed way.

The purpose of the research was explained and the purpose of each question was outlined. The team members were trained in ways of asking the questions. Some time was also taken to explain how to explain the questions in other languages when needed, especially into Krio or the mother tongue of the interviewee. Time was also taken to explain how to use the Kish Grid.

Each trainee was required to administer several test interviews and before being passed as suitable to conduct field interviews, the trainee had to satisfy us that he/she was capable and understood all instructions.

The questionnaire was not translated into Krio but the supervisor/recruiters ensured that all interviewers able to speak Krio and at least one local language of the area where the interviews were  to be conducted.

The interviews were conducted between 0700hrs and 1800hrs in the urban areas, but this period was extended further to 2000hrs in the rural areas. A note was taken of how long each interview took to complete. The shortest time was around10 minutes. The longest was just over one hour. The mean was 37 minutes.

Response Rate Analysis
A total of 2,099 questionnaires were completed. The following is an account of the success rate of interviewing the person selected using the Kish Grid, the number of repeat visits to find the selected person and the number of replacement interviews when the selected person could not be found. 
             

There was only one recorded refusal to be interviewed. Nobody was left out because of language problems, (all were able to understand the questionnaire) and the replacement rate for selected interviewees not being found was just over 12%. This means that nearly 88% of selected respondents were interviewed, a very high rate when compared to surveys anywhere in the world.

Quality Control

• Training and piloting was conducted to ensure that all the interviewers became full familiar with the survey instrument.
• Supervisors accompanied interviewer to at least some of their interviews to ensure that they were conducting the study as per the training they received. In the West Province, supervisors accompanied each interviewer to 14% of all their interviews. This is so because of the ease of accessibility to areas of their assigned location. However, in the provincial areas the situation was quite different. Supervisors accompanied enumerators to 8.2% of their interviews. The reasons being that the availability of transport and poor road networks make it very difficult to quickly access areas of supervision.
• In addition, the supervisor conducted a 10% back check to validate the answers recorded in the interview.
•  Questionnaires were checked in the field for consistency and completeness.

Sampling Outcome & Weighting
Sampling – More Detail
This survey was carried out using the method of sampling known as a stratified random sample.  The country was divided into its main administrative components, in this case its 14 Districts, divided between the 4 Provinces.  We then selected each administrative centre or town in each District as one sampled area. And then selected (using random numbers) areas or chiefdoms within each District. The number of such areas or chiefdoms depended largely on the population. Thus, in Freetown, we selected 20 areas, each with a sample of between  7 and 35, depending on population density. In the remaining Districts, between 3 and 7 chiefdoms in addition to the administrative town were selected, also using random numbers. The number to be interviewed in each selected chiefdom related to the population density and distribution.

The method of sampling is a probability method. This means that the standard statistical tests can be applied to any result. The following table shows the level of reliability of the results and the confidence we can place in them. This table provides confidence levels at the 95% level.

                                               
This is how to use the table. If the survey says that 40% listen to Radio station X, how reliable is this figure? We look for 40% - it is in the right hand column. We look at the column in the middle. We find the figure 2.2. This means that it is 95% likely that the real audience for Radio station X is 40% +/- (plus or minus) 2.2%. Or put in figures, Radio station X has an audience that is between 37.8% and 42.2%.

Weighting
With any kind of random sampling it is still possible to achieve a sample that has noticeable biases. This was the case with this sample. When we did the first analysis it was seen that the gender mix did not match that of the population. Also, it was somewhat over-educated. This happens sometimes in survey research. These particular variables, gender and education, are strongly correlated with media use. Men and the better educated are more likely to use media, especially radio, TV, the press and the Internet. It was important therefore to correct this bias to ensure that the results were not distorted in any way.

The sample had a bias towards males who made up 54% of the sample while the remaining 46% were female. The actual adult gender proportions in Sierra Leone are 47.5% males and 52.5% females.

Our achieved sample had too few people with no education and too many with secondary and higher. Therefore it was decided, as is often the practice in such cases, to apply weights to the data set. The weights used have corrected the biases and brought the data into line with what we know of Sierra Leone’s situation. Data on educational achievement in Sierra Leone were supplied by former colleagues at the BBC World Service who used these to weight the data from a survey conducted for them in Sierra Leone in 2007. The educational data used were from the Development Assistance Coordination Office (DACO/SLIS) in June 2006 The achieved sample showed a markedly higher level of educational achievement than we know to be the case with the adult population.

                       

These were the weights used in analysing the data:

                  

 [1] Population Reference Bureau,  2008 World Population Data Sheet, New York PRB, 2008.