Indicador de valor acrescentado e topicos sobre consistencia e estabilidade: uma aplicação ao Brasil

Maria Eugenia Ferrao, Alcino Couto

Resumo


Este artigo examina a abordagem de valor acrescentado no contexto da literatura de eficácia escolar. Debatidos aspetos relevantes do estado-da-arte sobre o uso do indicador de valor acrescentado, o enfoque recai sobre os problemas de escolha do modelo estatistico e o das propriedades de consistencia e estabilidade do indicador, considerando o contexto educacional brasileiro. Sao apresentadas as vantagens e limitacoes que, por sua vez, elucidam quanto ao uso que lhe possa ser dado. A componente empirica e desenvolvida atraves da aplicação de modelo multinivel de componentes de variância aos dados do GERES 2005 referentes ao municipio de Campinas, dado este apresentar, simultaneamente, maior percentagem de escolas por estrato e menor taxa de atrito. A evidencia empirica revela que o indicador produzido pelo modelo de resultados contextualizados e muito diferente do de valor acrescentado, sustentando a necessidade de adocao de estudos longitudinais. Os niveis de consistencia e de estabilidade encontrados face aos dois modelos de valor acrescentado testados sugerem a sua utilidade para atempadamente diagnosticar as escolas que sistematicamente apresentam desempenho proprio de escolas eficazes ou de escolas nao eficazes.

Palavras-chave


Desempenho escolar; Avaliação; Valor acrescentado; Modelo de regressao multinivel; GERES 2005

Referências


AITKIN, M.; LONGFORD, N. Statistical modeling issues in school effectiveness studies. Journal of the Royal Statistical Society: Series A, London, v. 149, p. 1-43, 1986.

ALLEN, R.; VIGNOLES, A. What should an index of school segregation measure? Oxford Review of Education, Dorchester on Thames, v. 33, n. 5, p. 643–668, 2007. DisponA­vel em: . Acesso em: mar. 2012.

ANDRADE, E. Alternativa de polA­tica educacional para o Brasil: school accountability. Revista de Economia PolA­tica, SA£o Paulo, v. 29, n. 4, p. 454–472, 2009.

ARIAS, R. M; SOTO, J. G. Concepto y evoluciA³n de los modelos de valor aA±adido en educaciA³n. Revista de EducaciA³n, Madrid, v. 348, p. 15-45, enero/abr, 2009.

BALLOU; D., SANDERS, W.; WRIGHT, P. Controlling for Student Background in Value-Added Assessment of Teachers. Journal of Educational and Behavioral Statistics, Washington, D.C v. 29, n. 1, p. 37–65, 2004. DOI:10.3102/10769986029001037

BALLOU, D.; SPRINGER, M. Achievement Trade-Offs and No Child Left Behind. Narshville, Tennessee, Vanderbilt University, 2008. DisponA­vel em : . Acesso em mar. 2012.

BIRD, S. M. et al. Performance indicators: good, bad, and ugly. Journal of the Royal Statistical Society: Series A, London, v. 168, n. 1, p. 1-27, 2005. DOI:10.1111/j.1467-985X.2004.00333.x.

BRAUN, H.; CHUDOWSKY; N.; KOENIG, J. Getting value out of value-added. Social Sciences. Washington, DC: National Academies Press, 2010. DisponA­vel em: . Acesso em: mar. 2012.

BRAUN, H.; WAINER, H. Value-Added Modeling. In: Rao, C. R.; S. Sinharay (Eds.), Handbook of Statistics: Psychometrics. Amsterdam: Elsevier, 2007, v. 26, p. 867–892) DOI: http://dx.doi.org/10.1016/S0169-7161(06)26027-9

BRIGGS, D. C.; WEEKS, J. P. The Persistence of School-Level Value-Added. Journal of Educational and Behavioral Statistics. Washington, D.C, v. 36, n. 5, p. 616-637, 2011. DOI:10.3102/1076998610396887

BROOKE, N.; BONAMINO, A. (Eds.). GERES 2005: razAµes e resultados de uma pesquisa longitudinal sobre eficA¡cia escolar. Rio de Janeiro: Walprint, 2011. DisponA­vel em: http://dc314.4shared.com/doc/DbyLSnul/preview.html.Acesso em : mar. 2012.

BROOKE, Nigel; SOARES, J. F. Pesquisa em eficA¡cia escolar: origem e trajetA³rias. Belo Horizonte: UFMG, 2008

CLOTFELTER, C. T. et al. Do school accountability systems make it more difficult for low-performing schools to attract and retain high-quality teachers? Journal of Policy Analysis and Management, New York, v. 23, n. 2, p. 251–271, 2004. DOI:10.1002/pam.20003

CRONBACH, L. Course Improvements through evaluation. Teachers College Record, New York, v. 64, n. 8, p. 672, 1963.

FERRAƒO, M. E. AvaliaA§A£o educacional e modelos de valor acrescentado: tA³picos de reflexA£o. EducaA§A£o & Sociedade, Campinas (SP), v. 33, n. 119, p. 455-469, 2012b.

FERRAƒO, M. E. Modelo de valor acrescentado: algumas evidAancias a partir do projecto 3EM, com discussA£o. CovilhA£: Universidade da Beira Interior, 2009b.

FERRAƒO, M. E. On the stability of value added indicators. Quality & Quantity, The Netherlands, v.46, n. 2, p. 627–637, 2012. DOI:10.1007/s11135-010-9417-6

FERRAƒO, M. E. Sensibilidad de las especificaciones del modelo de valor aA±adido: midiendo el estatus socioeconA³mico. Revista de EducaciA³n, Madrid, n. 348, p.137–152, 2009a.

FERRAƒO, M. E.; COUTO, A. The use of a school value added model for educational improvement: a case study from the Portuguese primary education system. School Effectiveness and School Improvement, accepted. 2013.

FERRAƒO, M. E.; GOLDSTEIN, H. Adjusting for measurement error in the value added model: evidence from Portugal, The Netherlands. Quality & Quantity, v. 43, n. 6, p. 951–963, 2009. DOI:10.1007/s11135-008-9171-1

FERRAƒO, M. E. et al. AferiA§A£o das aprendizagens em matemA¡tica no Ensino BA¡sico: a proposta 3EMat [Learning assessment in maths: the proposal 3EMat]. CovilhA£: Universidade da Beira Interior, 2009.

FERRAƒO, M. E. et al. A€ Procura da escola eficaz: referencial teA³rico do projecto de investigaA§A£o eficA¡cia escolar no ensino da matemA¡tica. CovilhA£: Universidade da Beira Interior, 2005.

FIELDING, A., YANG, M.; GOLDSTEIN, H. Multilevel ordinal models for examination grades. Statistical Modelling, London, v. 3, n. 2, p.127-153, 2003. DOI:10.1191/1471082X03st052oa

GOLDSTEIN, H. Methods in school effectiveness research. School Effectiveness and School Improvement, Bristal, v. 8, n. 4, p. 369-395, 1997. DOI:10.1080/0924345970080401

GOLDSTEIN, H. Multilevel statistical models, 4th ed. Chichester: Wiley, 2011.

GOLDSTEIN, H.; HEALY, M. The graphical presentation of collection of means. Journal of the Royal Statistical Society. Series A, v. 158, n. 1, p. 175-177, 1995.

GOLDSTEIN, H., & NODEN, P. Modeling social segregation. Oxford Review of Education. Dorchester and Thames, v. 29, n. 2, p. 225-237, 2003. DOI:10.1080/0305498032000080693

GOLDSTEIN, H.; NODEN, P. A response to Gorard on social segregation. Oxford Review of Education, Dorchester and Thames, v. 30, n. 3, p. 441– 442, 2004. DOI:10.1080/0305498042000260539

GOLDSTEIN, H.; SAMMONS, P. The influence of secondary and junior schools on sixteen year examination performance: a cross-classified multilevel analysis. School Effectiveness and School Improvement, Bristol, v. 8, n.2, p. 219–230, 1997.

GOLDSTEIN, H.; SPIEGELHALTER, D. League tables and their limitations: statistical issues in comparisons of institutional performance. Journal of the Royal Statistical Society: Series A, London, n. 159, v. 3, p. 385–443. 1996. DisponA­vel em: . Acesso em: mar. 2012.

GORARD, S. Comments on “Modeling social segregation†by Goldstein and Noden. Oxford Review of Education, Dorchester and Thames, v. 30, n. 3, p. 435-440, 2004. DOI:10.1080/0305498042000260520.

GRAY, J.; GOLDSTEIN, H.; JESSON, D. Changes and improvements in schools’ effectiveness: trends over five years. Research Papers in Education, Exeter, n. 11, p. 35-51, 1996.

GRAY, J.; GOLDSTEIN; H.; THOMAS, S. Predicting the future: the role of past performance in determining trends in institutional effectiveness at A level. British Educational Research Journal, London, n. 27, p. 391–405, 2001.

GRAY, J., et al. A multi-level analysis of school improvement: changes in schools’ performance over time. School Effectiveness and School Improvement, Bristol, v. 6, n. 2, p. 97-114, 1995.

HANUSHEK, E.; RAYMOND, M. Does school accountability lead to improved student performance? Journal of Policy Analysis and Management, New York, v. 24, n. 2, p. 297-327, 2005.

HANUSHEK, Eric A. The economic value of higher teacher quality. Economics of Education Review, Elsevier, Amsterdam, v. 30, n. 3, p. 466-479, june 2011.

HILL, P.; ROWE, K. Multilevel modeling of school effectiveness research. In: International Congress for School Effectiveness and Improvement, 7., Melbourne, 1994.

HOFMAN, R.; DIJKSTRA, N.; HOFMAN, W. School self-evaluation and student achievement. School Effectiveness and School Improvement, Bristol, v.20, n.1, p. 47-68, 2009.

HULPIA, H.; VALCKE, M. The use of performance indicators in a school improvement policy: the theoretical and empirical context. Evaluation & Research in Education, [s.l.], v. 18, p. 102-119, 2004.

KYRIAKIDES, L.; CREEMERS, B. P. M. A longitudinal study on the stability over time of school and teacher effects on student outcomes. Oxford Review of Education, Dorchester on Thames, v. 34, p. 521-545, 2008.

LADD, H. F.; WALSH, R. P. Implementing value-added measures of school effectiveness: getting the incentives right. Economics of Education Review, Elsevier, Amsterdam, v. 21, n. 1, p. 1-17, 2002. DOI:10.1016/S0272-7757(00)00039-X.

LECKIE, G.; GOLDSTEIN, H. The limitations of using school league tables to inform school choice. Journal of the Royal Statistical Society: Series A, London, v. 172, n. 4, p. 835-851, 2009. DOI:10.1111/j.1467-985X.2009.00597.x.

LOCKWOOD, J. R. et al. The sensitivity of value-added teacher effect estimates to different mathematics achievement measures. Journal of Educational Measurement, Washington, D.C., v. 44, n.1, p. 47-67, 2007. DOI:10.1111/j.1745-3984.2007.00026.x.

LOUREIRO, M. J. et al. AvaliaA§A£o do autoconceito Infantil. Revista Psychologica, Coimbra, v. 52, n.1, p. 469-478, 2001.

MARTINEAU, J. A. Distorting value added: The Use of Longitudinal, vertically scaled student achievement data for growth-based, value-added accountability. Journal of Educational and Behavioral Statistics, Washington (D.C.), v. 31, n.1, p. 35-62, 2006. DOI:10.3102/10769986031001035.

MCCAFFREY et al. Evaluating value-added models for teacher accountability. Santa Monica, CalifA³rnia, RAND Education, 2004. DisponA­vel em: . Acesso em: mar. 2012.

MORTIMORE, P. et al. The road to Improvement: reflections on school effectiveness. Lisse: Swets & Zeitlinger Publishers, 1998.

MORTIMORE, P. et al. School matters. Somerset: Open Books, 1998.

MORTIMORE, P., SAMMONS, P.; THOMAS, S. School effectiveness and value added measures. Assessment in education. principles, policy & practice, [S.l.], v. 1, n. 3, p. 315-332, 1994. DOI:10.1080/0969594940010307.

MURILLO, F. J. Una panorA¡mica de la investigaciA³n iberoamericana sobre eficacia escolar. Revista Iberoamericana sobre Calidad, Eficacia y Cambio en EducaciA³n, Bilboa, Espanha, v. 1, n. 1, 2003. DisponA­vel em: . Acesso em: mar. 2012.

MOLLER, J. School leadership in an age of accountability: tensions between managerial and professional accountability. Journal of Educational Change, New York, v. 10, n. 2, p. 37-46. 2009.

NEWTON, X. A. et al. Value-added modeling of teacher effectiveness: An exploration of stability across models and contexts. Education Policy Analysis Archives, Arizona, United States, v. 18, n. 23, p. 23. 2010. DisponA­vel em: . Acesso em: mar. 2012.

NICHOLS, S.; BERLINER, D. Collateral damage: how high-stakes testing corrupts America’s Schools. Cambridge, MA: Harvard Education Press, 2007.

NODEN, P.; GOLDSTEIN, H. A brief response to Gorard and Fitz. British Educational Research Journal, London, v. 33, n. 2, p. 273-274. 2007. DOI:10.1080/01411920701292906.

OECD. Measuring Improvements in learning outcomes. Paris: OECD Publishing, 2008. DOI:10.1787/9789264050259-en.

RAUDENBUSH, S. W.; WILLMS, J. D. The estimation of school effects. Journal of Educational and Behavioral Statistics, Washington, D.C, v. 20, n. 4, p. 307-335, 1995. DOI:10.2307/1165304.

REARDON, S. F.; RAUDENBUSH, S. W. Assumptions of value-added models for estimating school effects. Education Finance and Policy,

Massachusetts v. 4, n. 4, p. 492-519. 2009. DisponA­vel em: . Acesso em: mar. 2012.

ROBINSON, V.; LLOYD, C.; ROWE, K. The impact of leadership on student outcomes: an analysis of the differential effects of leadership types. Educational Administration Quarterly, [S.l.], v. 44, n. 5, p. 635-674, 2008.

RUBIN, D. B.; STUART; E. A.; ZANUTTO, E. L. A potential outcomes view of value-added assessment in education. Journal of Educational and Behavioral Statistics, Washington, D.C., v. 29, n.1, p. 103-116, 2004. DOI:10.3102/10769986029001103.

RUTTER, M., et al. Fifteen Thousand Hours. London: Open Books, 1979. p. 279.

SAMMONS, P.; HILLMAN; J.; MORTIMORE, P. Key Characteristics of Effective Schools: a review of school effectiveness research. London, Institute of Education, 1995.

SAMMONS, P; School Effectiveness: Coning of age in the 21 st Century.Lisse, Swets e Zeithing Publishers, 1999.

SAMMONS, P.; THOMAS, S.; MORTIMORE, P. Forging links: effective schools and effective departments. London: Paul Chapman, 1997.

SAUNDERS, L. A brief history of educational “value addedâ€: How did we get to where we are? School Effectiveness and School Improvement, Bristol, v.10, n.2, p. 233-256, 1999. DOI:10.1076/sesi.10.2.233.3507.

SCHEERENS, J.; BOSKER, R. The Foundations of Educational Effectiveness. Oxford: Pergamon, p. 347 , 1997.

SCRIVEN, M. The methodology of evaluation. lafayette. Ind.: Purdue University, 1196, p. 61.

TEDDLIE, C.; REYNOLDS, D. The international handbook of school effectiveness research. London; Routledge; New York: Falmer, 2000, p. 411..

TEDDLIE, C.; STRINGFIELD, S. Schools make a difference: lessons learned from a 10-year study of school effects . New York: Teacher; London: College Press, 1993, p. 272.

TEKWE, C. D. et al. An empirical comparison of statistical models for value-added assessment of schools performance. Journal of Educational and Behavioral Statistics, Washington, D.C., v. 29, n.1, p. 11–36, 2004.

THOMAS, S.; MORTIMORE, P. Comparison of value-added models for secondary school effectiveness. Research Papers in Education, London, v. 11, n. 1, p. 5-33, 1996. DOI:10.1080/0267152960110103.

THOMAS, S.; PENG, W.; GRAY, J. Modeling patterns of improvement over time: value added trends in English secondary school performance across ten cohorts. Oxford Review of Education, Dorchester on Thames, v. 33, n. 3, p. 261-295, 2007. DOI:10.1080/03054980701366116.

TOWNSEND, T. International handbook of school effectiveness and improvement. New York: Springer, 2007.

VICENTE, P. Plano amostral do projecto 3EM – eficA¡cia escolar no ensino da matemA¡tica [Sampling Design of 3EM Survey - School Effectiveness in Maths]. In: FERRAƒO, M. E., NUNES, C.; BRAUMANN, C. (Ed.). EstatA­stica: CiAancia Interdisciplinar. Lisboa: Sociedade Portuguesa de EstatA­stica, 2007. p. 849-858.

WEST, M. School-to-school cooperation as a strategy for improving student outcomes in challenging contexts. School Effectiveness and School Improvement, Briston, v. 21, n.1, p. 93-112, 2010. DOI:10.1080/09243450903569767.

WHOLEY, J. S. Evaluation - promise and performance. Washington,

D.C.: The Urban Institute. DisponA­vel em: . Acesso em: mar. 2012.

YANG, M.; WOODHOUSE, G. Progress from GCSE to A and AS level: institutional and gender differences, and trends over time. British Educational Research Journal, London, v. 27, n. 3, p. 245-267, 2001.


Apontamentos

  • Não há apontamentos.




Direitos autorais 2016 Revista Ensaio: Avaliação e Politicas Públicas em Educação

Licença Creative Commons
Este obra está licenciado com uma Licença Creative Commons Atribuição-NãoComercial 4.0 Internacional.

SCImago Journal & Country Rank