This page is intended to provide a summary of some of the quantitative techniques that we are using in this project. It also provides some links to papers we have written about our data analysis techniques.
Since this project has began we have refined and developed some of our ideas regarding the use of segregation indices in measuring the impact of markets upon schools in England and Wales. Some of our arguments and techniques may be useful to other educational researchers, but also to social scientists in general.
Our Technical Annex discusses in some length the approach that we decided to take at the beginning of the project with regards to measuring segregation between schools.
Since then we have written a number of other papers that may be interesting to you:
- Comments on 'Modelling social segregation' by Goldstein, H. and Noden, P. (2003) in Oxford Review of Education, 29, 2, 225-237 (Gorard 2003). This brief paper is a commentary on a piece of work recently published in the Oxford Review of Education (Goldstein and Noden 2002) which purports to create a multi-level model of the social segregation between schools in England 1994-1999. Segregation, as a measure of the (un)evenness of the distribution of disadvantaged students between schools, is an important characteristic of the school system, related to social cohesion and school effects (Gorard, Taylor and Fitz 2003). To read this response (Adobe Acrobat pdf files) please click here.
- Education and Social Justice: the changing composition of schools and its implications (Gorard, 2000), published by the University of Wales Press, contains a section relevant to this project, based on our early findings. While generally well-received (see for example the Thomas review in the British Educational Research Journal forthcoming) the book has attracted two critiques - one methodlogical and one political. Responses to these critiques have been sent to the authors, but we have decided to air these for those interested in the project. We hope that the deliberations emanating from the critiques will further clarify our approach and methodologies. You can access the reviews, responses and subsequent exchanges here: the Response to Goldstein ( for the British Journal of Educational Studies, forthcoming) and the Response to Thrupp (for the Journal of Education Policy, forthcoming) (both Adobe Acrobat pdf files).
- For a comparison of the use of the many segregation indices that are often employed in Social Science research please take a look at our Working Paper 37 (pdf file) from the Discussion page.
- We have also considered the effects of the level of anlaysis employed in the calculation of the segregation measure. This relates the modifiable areal unit problem (MAUP) often considered in geographical research. To see this analysis please take a look at our Working Paper 40 (pdf file) from the Discussion page.
- We have also begun to consider the relationship between residential segregation and school segregation. To look at some of our preliminary results and ideas please find our Working Paper 39 (pdf file) on our Discussion page.
- Lastly, we have also begun to look at the relationship between school segregation and school performances over time. To take a look at our Working Paper 38 (pdf file) please go to our Discussion page. Expect to see further analysis of this shortly.
FSM as a comparator
FSM remain a powerful indicator of school performance (e.g. Kelly 1996, Thomas et al. 1997, ESIS 1998, OFSTED 1998) as well as a useful proxy for family poverty. It is interesting in this regard that the FSM segregation index, unique to this study, has been shown to be significantly better as a predictor of performance than either raw-figures of eligibility or percentage eligible per school, each of which are in turn clearly better than measures based on take-up alone. For example, 88% of school-level variance was explained by FSM in Gorard (1998c), compared to 46% explained by FSM in Gibson and Asthana (1998a). FSM is the most consistently collected and powerful indicator of the social makeup of schools that is now available retrospectively to 1988.
During the period of this study (1988-1999) the only criterion for FSMs was Income Support (the successor to Supplementary Benefit). The Social Security Act 1986 (in force 1988) abolished the discretion of LEAs to allow FSMs for deserving cases, and deleted Family Credit (the successor to Family Income Supplement) as a criterion for eligibility. This may not have been fair, but it has the advantage for research of being unambiguous and consistently applied. FSM is the indicator of social and educational need most usually used by LEAs in allocating scarce resources (Smith and Noble 1995). However, there are other measures that can add to the FSM picture and give alternative estimates of any social movement. Among these are pupil gender, parental occupation, statements of special needs, ethnicity, stages of English, and performance at Key Stages 1 and 2. Unfortunately no LEA has yet been able to provide a complete history back to 1988 of most of these indicators in the way that they can for FSM. In addition, while prior attainment in Key Stage assessments may be a good baseline for assessing the later progress of individuals (Plewis and Goldstein 1997), it may not be suitable for the analysis of effects at the school level since it 'designs out' the continuing impact of local social background. As a proxy for any missing school level data, 1991 Census data will be used, disaggregated at ward/catchment area level. Unfortunately these local data suffer several defects for the purposes of this study, most notably their age and consequent inability to chart the continuous small changes in population characteristics.
Therefore while other indicators will be used as available, FSMs remain the main measure. In an era of raw-score comparisons between schools when increasing concern has been expressed about the apparent underachievement of boys at school, the changing gender balance between schools (obtained from the same forms as the number on roll and FSM) will be an interesting measure of potential market impact, as successful schools may seek to maximise their proportion of girls. The same data on the gender composition of schools will also be necessary as a step-level refinement of the school effects model.
The FSM segregation index used here (see below) overcomes some methodological problems in using free-school-meals (FSM) as an indicator of poverty. Some of these lie at the level of the compilation of official statistics. For example, in Wales, the Welsh Office STATS1 forms have asked schools for the number of pupils eligible for FSM every year, while the DfEE Form7 asked schools about FSM take-up on a particular day until 1993, and since then has asked for eligibility. Although figures are not available on the proportion of families potentially eligible for FSM who do not register for Income Support, and some LEAs suspect that there are individuals outside the system (Smith and Noble 1995), it is clear that eligibility is a much safer measure than take-up which could be affected by systematic regional variation such as special Asian dietary requirements. The change in record-keeping in 1993 makes regional comparisons and year-on-year comparisons more complex. Abrupt changes in the number of FSMs may be due to policy changes or changes in methods of collecting the statistics, as well as being produced by external 'social effects', such as the local economy or changes in patterns of school choice.
This is at least part of the reason why some writers have suggested caution in the use of free school meal as indicators of academic expectations (Plewis and Goldstein 1997), although there are suggestions that the effects of socio-economic background on performance are age-related, with FSM being a much better indicator for the older pupils used in this study (Audit Commission 1998). In addition, such limitations are generally addressed to the use of FSM as an indicator for individual performance, rather than for school-level patterns as is intended here (Gibson and Asthana 1998b). Moreover, the method of calculating the ratio of two ratios, used by Gorard and Fitz (1998), ameliorates most of these problems anyway. By converting the number of pupils eligible (or taking) FSM in each school, to a measure of how far that number is away from what would be the school's fair share of such pupils, the resulting index has the same metric and the same theoretical distribution whichever measure of social background or educational disadvantage is used. This makes cross-year and cross-border comparisons feasible. It is also true that despite the reservations above, the correlation between eligibility for FSM and take-up in any school is above +0.96 for the data collected so far.
The FSM segregation index
The segregation between schools in any unit of analysis such as a Local Education Authority is defined as the proportion of its students who would have to change schools for there to be an even spread of disadvantage between schools. More precisely, using FSM, the segregation index is:
sum(modulus(FSMschool - FSMlea * NORschool/NORlea))/ FSMlea/2
Once this calculation has been understood it can be seen to be precisely what is meant in general usage by the term 'segregation' of a social characteristic. It can be used to decide whether schools are becoming more or less mixed in terms of parental poverty, or any other indicator such as ethnicity, stages of English, or special needs (which can be substituted for eligibility for free school meals). Using this method of analysis, not only is there no evidence in South Wales of an increase in between-school segregation since 1988 when parents were given greater legal powers to choose secondary schools, to the great surprise of the investigators themselves there was evidence of the opposite. Segregation between schools in terms of all indicators tested and at any level of aggregation appear to have declined slightly.
At least part of the explanation for any apparent discrepancy between the repeated findings from previous studies of the process of choice and this 'political arithmetic' model is contained in the first report of the unfunded Cardiff study (Gorard and Fitz 1998). While it may be the case that a market system of schooling can be a stratifying process, giving greater advantages to some than others, it may also lead to less social segregation between schools than the status ante. It may well be the case then, and this will be rigorously tested in the proposed project, that what the new findings show is that whatever the limitations of market forces and choice, the stratifying effects of ‘selection by mortgage’ may have been a good deal worse.
Most areas examined in our unfunded research thus far show some evidence of a ‘starting gun’ effect which disappears in an established market (Gorard 1997), whereby some families, including, we hypothesise, those likely to be ineligible for free school meals appear to be quicker to respond to new rights in relation to school choice. Nevertheless the consequent increase in segregation may be temporary. It then disappears and the graph flattens out at a lower segregation index than previously (see for example the graph for Richmond). This graph therefore suggests that findings with regard to the stratifying effects of markets are sensitive not only to geographical/spatial variation but also to changes over time. Those studies made immediately after the introduction of policies to increase school choice, such as the Education Reform Act 1988 for example, may quite validly produce different findings from those of studies made some time later. This is an issue worthy of further investigation.
Last update: January 2001
Maintained by Chris Taylor
1. Audit Commission (1998) Local Authority Performance Indicators 1996-97: Education Services, Audit Commission
2. ESIS (1998) Welsh and English LEAs: results contextualised by % eligible for free school meals, report by Education Support and Inspection Service of Bridgend, Caerphilly, Merthyr and Rhondda Cynon Taff
3. Gibson, A. and Asthana, S. (1998b) School performance, school effectiveness and the 1997 White Paper, Oxford Review of Education, 24, 2
4. Kelly, A. (1996) Comparing like with like, Education, 187, 1
5. OFSTED (1998) Standards and quality in education 1996/97, http://www.official-documents.co.uk/document/ofsted/ciar/second.html
6. Plewis, I. and Goldstein, H. (1997) Excellence in Schools - a failure of standards, British Journal of Curriculum and Assessment, 8, 1
7. Thomas, S., Sammons, P., Mortimore, P. and Smees, R. (1997) Differential secondary school effectiveness: comparing the performance of different pupil groups, British Educational Research Journal, 23, 4