| Session Title: Macro-level and Micro-level Methodologies for Evaluating Education System Functioning in Afghanistan |
| Multipaper Session 351 to be held in Adams Room on Thursday, November 8, 11:15 AM to 12:45 PM |
| Sponsored by the International and Cross-cultural Evaluation TIG |
| Chair(s): |
| Edward Kissam, JBS International Inc, ekissam@jbsinternational.com |
| Discussant(s): |
| Roger Rasnake, JBS International Inc, rrasnake@jbsinternational.com |
| Jo Ann Intili, JBS International Inc, jintili@jbsinternational.com |
| Abstract: This session examines the evaluation research toolkit necessary to effectively track initiatives for strengthening education systems in developing countries--using Afghanistan as a case study. The presentations draw on panelists' analyses of two macro-level datasets: the 2005 Afghanistan National School Survey data and the 2003 NRVA, on their experience conducting micro-level community case studies in remote rural areas of the country, and their ongoing ethnographic research in a demonstration cluster school initiative. The presentations will show that the micro-level research provides crucial supplementation to the national assessment using UNESCO's EFA framework in order to guide effective education system reform. Recommendations will be presented regarding the types of capacity-building needed to assure reliable research, data collection, and analysis. |
| Challenges in Interpreting National Survey Data on Education: Moving From Summary Tabulation to Practical Action |
| Craig Naumann, JBS International Inc, cnaumann@jbsinternational.com |
| Shannon Williams, JBS International Inc, swilliams@jbsinternational.com |
| Edward Kissam, JBS International Inc, ekissam@jbsinternational.com |
| The presenters collaborated in detailed analyses of data from Afghanistan's Ministry of Education's 2005 National School Survey. These differed from previous analyses in that efforts were made to clean a dataset generated with limited resources and under difficult data collection conditions and to cross-tabulate key variables rather than simply generating national-level indicators of system status. The presenters will describe key findings from these analyses and their implications for assessing Afghanistan's progress in rebuilding an education system devastated by years of conflict. The discussion will include: strategies to monitor and respond to student dropout and teacher training initiatives to respond to dramatic variations from province to province in teacher qualifications, size of schools, and range of instruction provided. We will also present the team's recommendations for improved school survey design and practical issues to be addressed in strengthening the applied research capacity to reliably monitor national progress in education system reconstruction. |
| From Ritual Flowchart to Complexities of Real-world Action: Understanding the Local Community Context of School Functioning as an Element of Formative Evaluation |
| Mohammad Javad Ahmadi, Creative Associates International Inc, mohammadj@af.caii.com |
| Bianca Murray, JBS International Inc, bmurray@jbsinternational.com |
| Afghanistan's centralized command and control education system faces challenges in its efforts to effectively impact local instruction and student outcomes in a rural country with little infrastructure. Decentralization has been an important strand in both international donors' and Ministry of Education strategic planning. However, implementation of this macro-level strategy is problematic due to lack of information on variations in local conditions. This results in reliance on 'cookie-cutter models' for school administrator and teacher training and overall systemic change. The presenters describe their formative evaluation research in support of implementation of the first phase of a national initiative to strengthen local schools and quality of instruction via parallel training for school management teams and teachers. Evidence is presented that attention to variations in local conditions, to local problem-solving strategies, and to local perspectives on educational objectives, and different sorts of local resources contribute significantly to the design of promising decentralization initiatives. |
| Capacity-Building Challenges, Requirements, and Strategies for Strengthening National Education Systems' Evaluation Research Capacity |
| Trish Hernandez, JBS International Inc, thernandez@jbsinternational.com |
| Shannon Williams, JBS International Inc, swilliams@jbsinternational.com |
| Craig Naumann, JBS International Inc, cnaumann@jbsinternational.com |
| The authors describe and discuss the practical challenges inherent in efforts to develop the technical capacity of Afghanistan's Ministry of Education to conduct the applied research needed to effectively monitor progress in education system reconstruction and evaluate ongoing strategic initiatives. The discussion includes attention to the specific challenges and types of solutions needed to address problems at each stage in the evaluation research process: developing a baseline profile of system status, systematically identifying research priorities, formulating efficient and viable research strategies, creating workable sampling approaches in the absence of adequate sampling frames, generating and piloting study instruments, collecting, managing, cleaning, and analyzing study data, and reporting findings to decision-makers. |