Explaining CCRPI Results – High School, Part 2

In 2017, Douglas County high schools, collectively, earned 1.6 “Challenge Points” on the College and Career Readiness Performance Index (CCRPI) for the performance of “disadvantaged” populations: Economically Disadvantaged (ED), English Language Learners (ELL), and Students with Disabilities (SWD). This compares with 0.5 points for the state as a whole. Can we conclude from this result that these vulnerable populations are learning more and are better prepared for life after high school?

To answer this question, we need to first look at how the points are calculated. The calculation essentially is the product of the proportion of all students in one of these three categories (which determines the maximum possible points in this area) and the number of “performance targets” met by these subgroups. There are nine types of targets for each of the three groups: eight are EOC exam performance and one is graduation rate, resulting in 27 targets total.

The proportion of these three subgroups in Douglas County schools is very close to that in the entire state (60.7% compared to 62%). Hence, that variable in the equation has a minimal effect. Douglas County students, though, met 7 of 27 targets, whereas the State as a whole only met two:

Douglas County                                                                  Georgia

ED – Graduation Rate                                                         ED – 9th Grade Literature EOC
ED – 9th Grade Literature EOC                                        ED – American Literature EOC
ED – American Literature EOC
ED – Economics EOC
ELL – Graduation Rate
SWD – Graduation Rate
SWD – Geometry EOC

This difference of five targets met results in the positive differential of 1.1 points compared to the State. Yet notice that three of these five targets are graduation rates. Douglas County is graduating these subgroups at a higher rate than the State. The fact that there are two more EOC targets met may suggest to you that these students are learning more, and thus the higher graduation rate is legitimate, but what does the actual test score data reveal?

Table 1 shows the weighted EOC exam proficiency rates for these subgroups. In this formula, students who score at the “Proficient Learner” level are weighted 1.0, while those who score “Developing Learner” are weighted 0.5. Thus, a score of a 50 could mean that 50% were Proficient, or that 100% were Developing, or some combination thereof.

ED EOC Scores

Table 1 – EOC Performance and Graduation Rates of Disadvantaged Subgroups (bold means subgroup targets were met; green means outperformed the State, though targets were not met. 

Douglas County students had lower exam scores in 18 of 24 areas, yet the graduation rates were 7 to 11 points higher for each group. We can conclude from these results that more students graduated while achieving less, which further confirms that the district’s higher graduation rates are NOT the result of higher achievement but of lower academic standards. Furthermore, since Proficiency on these exams is the level indicating that students are ready for college-level work, it is also the case the fewer of these students are prepared for college.

Perhaps this is why fewer Douglas County graduates, as a whole, are enrolling in college in recent years. For example, according to the Governor’s Office of Student Achievement, 64% of 2015 high school graduates statewide enrolled in college (including technical schools); whereas Douglas Co. graduates enrolled at a 58% rate.

It is reasonable to conclude then that these subgroup graduation rates are inflated relative to actual achievement. Together they result in 0.7 more Challenge Points. Combined with the fact that the TOTAL graduation rate, which includes all students and is thus likewise made higher by these subgroups’ rates, results in at least 1 more Achievement Point, then altogether questionably high graduation rates are the cause of at least 1.7 additional CCRPI points.

This small amount is not trivial. It accounts for over half the positive differential with state high school CCRPI, and it allows, potentially, for high schools to avoid accountability under the district’s “Strategic Waiver” contract with the GA DoE. This consequence is especially galling considering the fact that the waivers include regulations affecting students who are ELL and SWD. For example, the school system can now enlarge class sizes for these groups to whatever size they wish and still receive additional funding for them, whereas prior to the “waivers” this funding was contingent on staying below certain class size maximums.

This situation reeks of injustice. If our high schools, and thus the administrators who run them, look better from higher graduation rates resulting in higher CCRPIs, and these higher scores come at the expense of properly educating the most vulnerable students in the school system, then people in power are benefiting at the expense of the poor and marginalized. Is exploitation taking place in our high schools?

This is the final post in a series on 2017 CCRPI Results. See also:

Elementary CCRPI

Middle School CCRPI

High School CCRPI Part 1

Explaining CCRPI Results – High School, part 1

 

high school

In contrast with the elementary school and middle school CCRPI, Douglas County’s high school CCRPI increased in 2017 (by 1.3 points) and also beat the state CCRPI (by 3.2 points) . In view of persistently low levels of high school student achievement in Douglas County and the fact that many recent graduates are not doing well in college , how can we account for this result?

Table 1 breaks down the main components for CCRPI for the District versus the State.  Douglas County scores exceed the state by one point or more in Achievement (a composite measure of success from a single year); Progress (basically the change in EOC test scores from the previous year relative to groups of similar students across the state); and Challenge (up to 10 extra bonus points earned through a variety of means). Let’s analyze each of these in order.

HS CCRPI t1

‘Achievement’ is comprised of 18 indicators, which are grouped into 3 categories: ‘Content Mastery’ (#1-8) is a weighted average of passing rates on the eight EOC exams and counts for 40% of the points; ‘Post High School Readiness’ (#9-16) includes a variety of indicators predictive of career and/or college success and counts for 30%; and ‘Graduation Rate’ (#17-18) includes both the 4-year and the 5-year rate and counts for 30%. Table 2 shows the relative weight of each indicator, and compares the points Douglas County earned on each with the State.

The eight indicators in which the District outperforms the State are in bold (see Indicator Key at bottom). Three are EOC exam scores; two are graduation rates. Of the remaining three in Post High School Readiness, two (#s 12 and 13) have to do with completion of courses or a sequence of courses, and not with achievement assessed independently of the schools, which means they can be driven up by lowering standards and inflating grades. For example, we know that nearly 100% of students pass AP courses in Douglas County, even though only 10-15% take and pass AP exams. This high course pass rate is the main reason why the district scores high on indicator 12.

Notice the contrast between indicator 12 and indicator 11, which is called the “College Readiness” Indicator. It is the percentage of graduates meeting at least one of a list of achievement known to predict for success in college. Only 63% of Douglas County graduates qualified, compared to 73% for the state, yet more are passing “accelerated courses” like AP.

HS CCRPI t2

To assess the impact of these indicators on the overall CCRPI, it is helpful to look at the relative contribution of each of the three categories. Table 3 shows the “weighted achievement’ of each category.  Douglas County earned 71% of the possible achievement points compared to 69% from the state, resulting in a 1.1 point differential due to higher scores in Post High School Readiness and Graduation Rate. How much of this difference is due to REAL achievement?

HS CCRPI t3

Indicator 10 requires students to pass an “end of pathway assessment.” These are technical skills tests linked to industry standards. I know very little about what these entail, but if they are indeed assessed by experts external to the schools, this would be a legitimate and significant achievement. However, as already explained, Indicators 12 and 13, which together account for over 2/3rds of the 0.85 point difference, merely require completion and thus can be driven up by manipulating/inflating grades.

As for Graduation Rates, we know that these are too high relative to real measures of academic achievement. If graduation rates were proportional to real achievement, the Douglas County’s rates would be lower than the state’s. Thus, at least 1 point of the CCRPI difference is due to Douglas County’s inflated graduation rates.

Finally, the Challenge Points are also 1.1 points higher than the State. Challenge Points have two sub-categories: the performance of disadvantaged students and “exceeding the bar” points that can be earned in nine different ways. Both the district’s and the state’s Challenge Points were earned in the former category. Does this mean, then, that Douglas County’s disadvantaged students (economically disadvantaged, English language learners, and students with disabilities) are doing better than disadvantaged students throughout the state?

That is a complicated question, and will be the focus of Part 2 of this post.

INDICATOR KEY

1 – Weighted percent of students scoring at Developing Learner or above on the Georgia Milestones Ninth Grade Literature EOC (required participation rate >= 95%)
2 – Weighted percent of students scoring at Developing Learner or above on the Georgia Milestones American Literature EOC (required participation rate >= 95%)
3 – Weighted percent of students scoring at Developing Learner or above on the Georgia Milestones Algebra I/Coordinate Algebra EOC (required participation rate >= 95%)
4 – Weighted percent of students scoring at Developing Learner or above on the Georgia Milestones Geometry/Analytic Geometry EOC (required participation rate >= 95%)
5 – Weighted percent of students scoring at Developing Learner or above on the Georgia Milestones Physical Science EOC (required participation rate >= 95%)
6 – Weighted percent of students scoring at Developing Learner or above on the Georgia Milestones Biology EOC (required participation rate >= 95%)
7 – Weighted percent of students scoring at Developing Learner or above on the Georgia Milestones US History EOC (required participation rate >= 95%)
8 – Weighted percent of students scoring at Developing Learner or above on the Georgia Milestones Economics EOC (required participation rate >= 95%)
9 – Percent of graduates completing a CTAE pathway, or an advanced academic pathway, or an IB Career Related Programme, or a fine arts pathway, or a world language pathway within their program of study
10 – Percent of graduates completing a CTAE pathway and earning a national industry recognized credential (passing an end of pathway assessment)
11 – Percent of graduates entering TCSG/USG not requiring remediation or learning support courses; or scoring at least 22 out of 36 on the composite ACT;
or scoring at least 480 out of 800 on Evidence-Based Reading and Writing and 530 out of 800 on Math on SAT; or scoring 3 or higher on two or more AP exams; or scoring 4 or higher on two or more IB exams
12 – Percent of graduates earning high school credit(s) for accelerated enrollment via Move on When Ready, Advanced Placement courses, or International Baccalaureate courses
13 – Percent of graduates completing a career-related Work-Based Learning Program or career-related Capstone Project (Includes IB projects)
14 – Percent of students achieving a Lexile measure greater than or equal to 1275 on the Georgia Milestones American Literature EOC
15 – Percent of students’ assessments scoring at Proficient or Distinguished Learner on Georgia Milestones EOCs
16 – Percent of students missing fewer than 6 days of school
17 – 2017 4-Year Cohort Graduation Rate (%)
18 – 2016 5-Year Extended Cohort Graduation Rate (%)

 

 

 

 

Explaining CCRPI Results – Elementary School

kids

Douglas County’s Elementary Schools’ CCRPI dropped by a stunning 8.4 points from 2016 to 2017, and is now below the state average (69.9 compared to 72.9) for the first time in a few years. Why did it drop so precipitously and why is it three points below the state?

The first table in the spreadsheet below compares the major CCRPI components from 2016 and 2017. Achievement (a composite measure of success from a single year) remained essentially the same; Progress (basically the change in test scores from the previous year relative to groups of similar students across the state) declined by 2.7 points; Achievement Gap (the success or progress of the lowest 25% of students relative to state norms) declined by 1.7 points; and the Challenge points (up to 10 extra bonus points earned through a variety of means) dropped from 4.1 to zero.

Since the Challenge points fell the most, let’s look more closely at what that means.

In 2016, the district earned its 4.1 challenge points because its disadvantaged populations (Economically Disadvantaged, English Language Learners, and Students with Disabilities) met their test score targets in 7 of 12 possible areas (this ratio is multiplied by the proportion of the total population classified in one of these categories). But in 2017, no test score targets were met by these groups! Specifically, the declines were in Math, Science, and Social Studies. Since these groups of students tend to dominate the lowest 25% of all students, this is also why the Achievement Gap fell.

That accounts for most of the decline. Compared to the state in 2017, we see that Achievement, Achievement Gap, and Challenge Points were lower. The second table breaks down the Achievement points. The biggest gulfs here with the state are test scores: we have more students failing tests, except for Science (indicators 1, 2, and 4) and fewer students are scoring at the top two levels (indicator 12; which means fewer finish the year at grade level!).

It should concern us all deeply that only 38% of our Elementary students are on grade level in any given subject. The numbers are alarming also for the state as a whole. This does not bode well for the future.

The power to change this lies in the local community. We have to hold school leaders accountable, while also recognizing that educating children is the work of the whole community. It will take a collective efforts by parents, grandparents, business leaders, in concert with the school system to bring about major improvements.

 

 

Are our Disadvantaged Students Really Learning? – an Analysis of the 2017 US News High School Rankings

download

The latest US News & World Report High School Rankings (2017) indicate that disadvantaged populations of students are underperforming in Douglas County, and that high graduation rates have been achieved in large part by expedient means of moving such students through the system. This post will explain briefly the ranking’s methodology and then analyze some of the data to support this conclusion.

Only 1 of 5 high schools – Douglas County High – qualified for a ranking. To qualify for the lowest ranking – “Bronze” – schools must meet two criteria. First, their students as a whole must perform better on reading and math exams, administered by the state, than what is statistically expected based on the school’s poverty level (typically measured by the % of the student body on free and reduced lunch).  Second, the “disadvantaged” student groups (includes African-American, Hispanic, and poor students) must perform better than the state average for those populations.

To qualify for “Silver” or “Gold” ratings, there is an additional criteria: schools must exceed the median College Readiness Index (CRI). The CRI is comprised of both AP participation (how many students take an AP exam) and AP performance (how many students pass an AP exam). AP exams are used since these are college level exams through which students can earn college credit by passing.

To understand why 4 of 5 high schools did not qualify for a ranking, let’s look at Alexander High School as a case study (AHS consistently has the top CCRPI score in the district). In reading and math, Alexander students underperformed statistical expectations, having a gap score of negative 18 (a negative score means the actual performance was lower than expectations). Also, its disadvantaged populations scored 8.9 percentage points below the averages for disadvantage students throughout the state (only 12.7% of disadvantaged students were proficient in reading and math).

To understand why Douglas County High School earned a Bronze but not a Silver or Gold, let’s look at its statistics. Its gap score was positive 6.8, and its disadvantaged populations scored 6.5 percentage points above the state average. Thus, the first two criteria were met. Given the fact that its students with disabilities reading and math scores were on par or worse than the district as a whole, it is reasonable to assume that these test scores were boosted by the school’s population of International Baccalaureate students (at least 25% of test takers), which sees significant participation from African-American students.

DCHS did not qualify for Silver, though, because of low performance on AP exams. Even though the participation rate was fairly high (39%), the passing rate on all exams was only 17%, with 28% of seniors passing at least one exam (this is higher than 17% because it only requires one passed exam and excludes multiple failed exams). It is worth noting that in years past, DCHS ranked higher (recall the top 10% ranking?) because the rankings used IB exams instead of AP.

How does this relate to the high graduation rate spike in 2015? These latest rankings use 2015 academic data.  The fact that disadvantaged students are performing less than their peers while graduating at higher rates indicates clearly that the standards for graduation were lower. The high participation of disadvantaged students in the county’s suspect online credit recovery program, e2020, further supports this conclusion.

Stopping for the School Bus: A Cautionary Tale

passing a school bus

In February, a member of the Douglas County Board of Education received a ticket for illegally passing a school bus, which was stopped, lights flashing, to either drop off or pick up children (Douglas County Sentinel, March 19th 2017). The BoE member, through his lawyer, expressed regret, saying that the misdemeanor crime was committed “inadvertently.” Given that the road on which this happened has four lanes with a turn lane, I think he deserves the benefit of the doubt; I can see this happening to the best of us. But this incident is pregnant with deeper meaning. I hope our school system’s leaders will recognize the symbolic significance of this event and learn from it.

Why is passing a stopped school bus not only illegal but wrong? Negligent drivers risk the well-being of students coming off a bus in their haste to reach a particular destination. We recognize instinctively that the few seconds that the driver saves on his commute time are not even comparable to the risk of putting children’s safety in danger. When a driver does this, we are rightly aghast that he would judge this risk to be worth a few seconds of his time. The crime is especially galling when committed by someone whose public duty is to ensure that our children’s well-being is the top priority of the school system.

Knowing why this behavior is wrong allows us to see in it a poignant analogy for other wrongs in the school system.  On its way to reach such worthy destinations as high graduation rates and increased participation in AP courses, the school system has risked the well-being of our children by pursuing them in a hasty, careless manner. Serious long-term risks to our children’s futures have been taken for the sake of short-term benefits to the adult drivers.

These goals have been achieved by expedient methods such as as cheating in online credit recovery courses and aggressively recruiting students who are not yet proficient in high school level work into what are supposed to be college-level courses (AP). The result of this careless haste is to weaken the integrity and thus value of a high school diploma, and to impede achievement in AP courses by lowering academic standards to accommodate unprepared students. Consequently, many graduates are ill-prepared to succeed in college or lack the job skills needed for a productive career.

I don’t believe that the BoE member in this case consciously thought, “I don’t care about these children; I just want to get to where I am going as fast as possible.” This wasn’t an act of malice; it was an act of neglect. Similarly, our school system’s leaders are not callous towards children’s well-being, but there has been negligence in considering carefully the long-term consequences of their means and methods. The ends of high graduation rates and high participation on AP courses do not justify the means of attaining them.

Our school system needs to slow down and pay more attention to how the shortcuts they have taken to reach these goals are putting our children’s futures at risk.