Texas Education Agency
Evaluation of the Texas SUCCESS Initiative
The Texas Students Using Curriculum Content to Ensure Sustained Success (SUCCESS) program offers state-funded access to computerized interactive mathematics and reading programs. These programs are provided by two vendors—Istation Reading (Istation) and Think Through Math (TTM)—to all Texas public school students in Grades 3-8.
We employed a mixed-methods approach to evaluate the Texas SUCCESS program that had been implemented in school districts across the state. Our design approach was quasi-experimental and explored the relationship between the intensity of Istation and TTM program usage and student outcomes on the 2013-14 State of Texas Assessments of Academic Readiness
Generally, a positive relationship was observed between the completion of online math lessons and gains on the 2013-14 STAAR® test; however, results were mixed depending upon the program and grade level.
To access the full report on SUCCESS visit the Texas Education Agency’s website:
Texas Higher Education Coordinating Board
Evaluation of the AVID for Higher Education Program
The AVID for Higher Education (AHE) program is designed to provide students with the supports needed to be successful in college. Our evaluation of the AHE program spanned three years and included an extensive assessment of program implementation. We made a comprehensive analysis of the relationship between program participation and key student metrics (persistence in college, cumulative grade point average, and cumulative credit accrual).
The study, which tracked student outcomes for three cohorts of students (fall 2012, fall 2013, and fall 2014), found small, yet statistically significant program effects. Click here to access the full report.
New Hampshire Department of Education
Statewide Parent Involvement in Special Education (Indicator 8) and Post School Outcomes (Indicator 14) Surveys
Our research team works with the NHDOE to annually survey two target populations for required state reporting:
- parents of students receiving special education services (to measure schools’ facilitation of parent involvement as a means of improving services); and
- students with an Individualized Education Plan who exited the school system in a given year (to measure post-high school employment and education paths).
On an annual basis, Gibson administers surveys to parents of over 30,000 students and to more than 3,000 exited students across the state. We use Rasch scaling methods on the NCSEAM Parent Involvement surveys to calculate results and to provide district-level reports in addition to statewide results. We also provide technical support to school districts in understanding and using results to make targeted improvement efforts.
Click here to access a statewide summary report from this project.
Texas Higher Education Coordinating Board
Evaluation of the Comprehensive Student Success Program
Implemented in eight higher education institutions across the state, the CSSP had several goals: increase student completion of courses with traditionally high rates of failure and/or withdrawal, increase completion of degrees or certificates, improve faculty involvement in student success initiatives, and increase student participation in campus activities and events. Institutions that participated implemented early warning data systems, made changes to instructional approaches, increased contact between advisors/counselors and students, and increased faculty involvement in supporting student success.
Gibson employed a mixed-method design incorporating interviews and classroom observations to measure implementation, with quantitative analyses examining program outcomes. Results were promising, with CSSP students earning more credits on average than comparison students and demonstrating higher rates of persistence. Results varied by institution.
Tyler Independent School District
Evaluation of the Bilingual/English as a Second Language Program
The Gibson research team, along with experts in bilingual/ESL education from American Institutes for Research (AIR), conducted an evaluation of Tyler’s bilingual/ESL program in an effort to identify areas for focused, targeted improvements.
This study included two main components:
- Analysis of the districts’ English Language Learner student data (including assessment scores, language proficiency scores, parent denial rates), including examination of performance over time by various sub-groups of interest (e.g., by grade, by school, by ELL status).
- Examination of the district’s implementation of its bilingual model, and the identification of areas where improvements may have impacts on student outcomes.
Various strengths of Tyler’s programming were identified, and recommendations were made for program improvement, particularly in the areas of program model selection and implementation, as well as collaboration and coordination across departments, strategies for improving identification and placement, and ideas for attracting/developing qualified bilingual staff in needed areas.
Click here to read an article about the program.
United States Department of Education
Research Projects under the Regional Educational Laboratory Southwest
The Gibson research team also works on various projects funded through the Department of Education. These studies span various states, including Texas, New Mexico, and Virginia, and are conducted in partnership with Regional Education Laboratories.
These projects include studies that focus on:
- The under-representation of Hispanic students in Science, Technology, Engineering,
and Math (STEM) courses (while in high school) and fields of study (in college).
- Advanced high school course-taking patterns across different student groups—particularly historically underserved student groups—and advanced course offerings across different schools.
- Student performance in low-performing schools with various accountability designations.
- Longitudinal participation rates in state-funded PreK programs disaggregated by student
- Kindergarten readiness and psychometric assessment of school readiness measures.
Click here to learn more about the Regional Educational Laboratory Southwest.
Austin Independent School District
Risk Assessment and Long-Range Plan
In conjunction with the Austin Independent School District’s Internal Audit department, Gibson performed a comprehensive risk assessment for Austin Independent School District as part of the pilot phase of the partnership between the Texas Association of School Boards and Gibson.
The project included over 40 interviews with Austin Independent School District staff and members of the Board of Trustees. We assisted the Internal Audit department in developing its long-range audit plan and FY 2012-13 audit schedule. Austin Independent School District contracted further with Gibson to help the district transition its internal audit function from its pre-existing narrow focus towards an expanded scope of coverage that included all programmatic and operational areas of the school system.
Hays Consolidated Independent School District
Academic Program Management Audit
Conducted as a part of our long-term internal audit agreement with Hays Consolidated Independent School District, this audit focused on the management processes, school-level practices, automated tools, and the accountability mechanisms used to support the academic program at Hays Consolidated Independent School District.
We analyzed six major areas of academic management including curriculum implementation fidelity, student data analysis and use, instructional resources and support, teacher observation and evaluation, teacher professional development, and academic program management and decision-making.
Our approach involved the analysis and the corroboration of data from multiple sources including district-provided data, teacher surveys, interviews with central office staff, teacher and principal focus group sessions, and classroom observations.
Our recommendations included changes to academic program organization structure, decision-making framework, student data analysis, classroom observation protocols, lesson plans, teacher professional development, and procedures through which curriculum implementation is monitored.
Fort Bend Independent School District
Student Information System Internal Audit
As a part of our long-term internal audit agreement with Fort Bend Independent School District, Gibson conducted an audit of the district’s student information systems. Our review covered policies and procedures governing the collection, processing, maintenance, and reporting of student information during the 2013 and 2014 school years.
We also reviewed operations and controls related to the district’s primary student information system, and performed data analytics on the entire student database to identify issues and their magnitude. Areas of improvement were identified and recommended in the final report.
Killeen Independent School District
Purchasing and Accounts Payable Audit
Conducted as a part of our long-term internal audit agreement with Killeen Independent School District, Gibson conducted an internal audit of the purchasing and accounts payable departments for the district. The audit involved interviews with purchasing department and accounts payable department staff as well as data analytics and testing of transactions to ensure compliance with board policies, administrative regulations, and district operating procedures. The report included tighter controls over competitive bidding and improved procedures for processing purchase orders.
Texas Association of School Business Officials
For the Texas Association of School Business Officials, Gibson upgraded a comprehensive statewide student, financial, staffing, and demographic database – called the Financial Analysis and Comparison of Texas Schools (FACTS) – to include data visualization tools accessed through the Internet. This upgraded business intelligence tool – eFACTS+ – provided multi-dimensional online analytical processing functionality in a fully accessible web-based environment, which helped school systems’ analysts easily analyze performance trends via a researcher friendly interface fit for the casual and power user.
In the years following the upgrade, Gibson provided further eFACTS+ enhancements as well as training.
School District of Lee County
Community Survey and Data Visualization Tool
Gibson worked with the School District of Lee County on a large-scale community survey, as well as a survey of secondary school students, assessing attitudes and opinions about student assignment approaches, the district’s programs and services quality, expanded program preferences, and parent involvement issues.
As a part of that project, Gibson developed an interactive data visualization tool that graphically presented results from the survey, allowing in-depth examination of survey responses based on any demographic characteristic of interest (e.g., ethnicity, gender, and income). With the click of a button users can choose to display survey results by respondent demographics to examine results that interest them.
Manor Independent School District
Budget Analysis Tool
Manor Independent School District hired Gibson to assist the district in analyzing its 2015-16 budget. Gibson helped the board to better understand Manor Independent School District’s expenditure and staffing levels and to better answer specific questions about budget line items by developing a budget analysis and data visualization tool.
In collaboration with the board, Gibson collected and validated the district’s expenditures and staffing data. Working with the board and senior leadership, we designed and developed a visualization tool that allowed users to analyze expenditure data in the aggregate and on a per student basis. The tool also allowed users to drill down according to every element of the district’s account code structure, including locally assigned codes.
Jeffco Public Schools (Colorado)
Employee Satisfaction Survey and Data Visualization Tool
For Jefferson County Public Schools (JEFFCO) – the largest school district in Colorado – Gibson performed cross-sectional and longitudinal data analysis on the school system’s employee satisfaction survey data. We analyzed 10 years of employee data to examine differences between employee groups and trends over time for different areas of satisfaction – including constructs such as employee effectiveness, district diversity, supervision, compensation, and benefits.
In addition to providing JEFFCO with targeted recommendations to improve employee satisfaction in key areas, we developed an interactive data visualization tool for JEFFCO that enabled leadership to view survey results at the construct level or item level, and to slice mean scores by any of the demographic characteristics available for analysis.