Data-Driven Transparency


I once had a college professor tell me that 70% of all statistics were made up on the spot...except for this statistic, of course. I don't know for sure if quite that many statistics are made up on the spot, but I certainly can vouch for the fact that data is often engineered, filtered, and manipulated in order to produce the results people are looking for. After being an administrator for over 11 years in OCPS, I trust data even less than I did during my college years. I've found that there's a lot more behind the data that the general public may never see, and many times it's what the data doesn't show that truly matters most. I am not a proponent of high-stakes testing or grading schools using comparative student data, but I do believe in being transparent with the public. I understand why school districts manipulate and engineer data in such a high-stakes environment, but I don't think we should oversell our successes when, in many cases, we've just learned how to "game" the system. Allow me to explain how data can be manipulated and engineered.

As a hypothetical example, when the state introduced the Biology EOC a few years back, the pass rate on the exam became part of a school's grade. One school shrewdly decided to only enroll their honors students into the Biology class that year in order to guarantee that they would have an extremely high pass rate. The other students in that grade who were "less-than-honors" students were probably put into a Physical Science class that may or may not have counted toward college acceptance or bright futures' calculations--I'm not exactly sure, but I would love to see the data on this fictitious example. The high school led the district in Biology pass rates, the principal received recognition for a job well done, and the principal's career as a data guru was secured forever. But did anything amazing really happen that justified an award? Or was it just a matter of manipulating the data...or not including what the data didn't explain--all the other schools included their lower-level students, which naturally decreased their pass rates. To me, manipulating data is just as bad as making data up out of thin air. In a data-driven era, those who know how to manipulate data are exalted. The question I have is this, "Was enrolling only the top level students into the Biology class really what was best for the school or the students? We are supposed to be driven by what is best for our students, not what is best for the school and the data. Those students who did not take Biology that year ended up taking a class that they probably didn't need for graduation.

2. Another "hypothetical" example of data manipulation would be how some schools can theoretically improve the data on the growth of their lowest 25% students--as measured by the old FCAT Reading and Math tests. With the new advent of alternative schools popping up around traditional high schools--some examples would be Drop Back In, Sheeler High School, and Sunshine Charter High School--one hypothetical way that traditional high schools can improve the growth rate of their lowest 25% of their students is to simply get their lowest 25% off their books by aggressively counseling and withdrawing those low students and guiding them to the local "credit recovery", alternative high schools in their area. If you remove a good percentage of your lowest 25%, a good portion of your new lowest 25% is actually your 26-50%, which means your school just received a natural growth bump in your lowest 25% without actually doing anything but getting your lowest level students off of your books. This hypothetical strategy can be used to show how great a district is doing to make progress in traditionally struggling schools. I would like to see the data on how many times this strategy has been used. I think this data got left off of the report. This can also help traditional high schools with their graduation rate, and probably explains why OCPS has their graduation rate in the 90% range, yet the State of Florida has OCPS's graduation rate in the 77% range. Now, understand, many students are not finding success in traditional high schools, and they need a change of venue, but we shouldn't brag about the great success and progress we are making if all we did was get rid of our lowest students. Great success would involve motivating, inspiring, developing, and changing the lives of the students in our care, not simply getting them off our books.

3. Another area of "hypothetical" data manipulation is in the area of student participation in Advanced Placement courses--known as AP Participation. One way that a state, district, and school can improve their image is to increase the number of students taking AP courses and corresponding AP tests. Regardless of whether or not the students pass the tests, the school will receive AP participation points for students being enrolled in those classes. The ability of the district to earn participation points naturally encourages districts to enact policies or "guidelines" that steer students into AP classes, whether the students or parents desire to take college level classes or not. I've heard it said that just the act of taking an AP class will better prepare a student for future college courses. I don't disagree with this thought outright, but I do question if the person or organization making this statement is profiting from the massive increase in testing. I know this would certainly be a great resume builder for those who want to climb to the top of the education ladder, but is it what is best for students...even individual students? Within schools in this hypothetical district, it has become a standard practice to look at students' PSAT scores and enroll students in AP classes simply based on their AP Potential that can be determined by their scores on the PSAT. It literally takes close to an act of Congress to register a student for a course lower than their "AP Potential". I believe in stretching students in order to help them grow, but I also believe in allowing students to intellectually develop at a rate they are comfortable with...and to not overload them. I remember one student who filled out a course selection form and one of the classes he signed up for AP Psyfology...that's the way he spelled it. I signed him up for AP Intensive Reading. My daughter graduated last year from Apopka High School. She is a very bright student--she took 7 AP classes during her high school years at Apopka HS, scoring three 5's and four 4's. The "hypothetical system" tried to enroll her in five AP classes during her senior year. She only wanted to take three. College students typically only take 4-5 classes for a full load in college, yet we are forcing our high school students to take 5 AP classes and then maybe 2 other honors classes? That's lunacy...that's a recipe for disaster for many students. The sad thing is, if you schedule enough students this way, you end up making many other students and parents feel like they are falling behind--"You didn't get your AA degree when you graduated from high school? You didn't finish half of your high school classes in middle school? You didn't finish half of your middle school classes during Elementary school? What's wrong with you? You're falling behind the best?" The pressure that we put on our students is immense, and it is often released in negative ways...in senior pranks and negative behavior during "senior walk-outs" on the last day of school.

Now, the data that is often left off during many data driven decision meetings is how many students are being pushed into these AP experiences and they are receiving "F's" in those classes? I've seen far too many. An AP student should never receive an "F". Students taking AP classes should at least be able to get a "C", I would think. But understand, if you overload a student with upper-level classes, you have cooked up a disaster for that student and their family. What happens in many cases is schools will transfer students out of those classes in the Spring semester if the students are completely doing an "academic face-plant" in these AP courses. Schools do this to salvage the student's credits, especially if it is during the student's senior year. Legally, a student should have to repeat the course that they received an "F" in...I'm not sure this always happens. I think this type of data is often left off during our data-driven meetings. It is sad that we no longer trust professional school counselors to do their jobs in terms of guiding students according to their abilities, with the input of a child's current teachers. We really don't trust anyone to do their jobs any more. Now, I should add, I certainly think a student should be able to select any AP course they want to take, if they believe they have what it takes to succeed in such a class. That being said, we also need to keep the intelligent, experienced, skillful guidance of a child's counselor near the center of their academic development and academic progression plan. AP participation points, among other things, played a part in helping OCPS win the Broad Prize and get a handful of schools ranked by the U.S. News and World Report publication. If your student was one of the lucky ones who ended up with an "F" in an AP class, now you know part of the behind the scenes motivation for such a push. I would love to see the data on how many "F's" were given to AP students, or how many had to be transferred out of those classes during the 2nd Semester. What the data doesn't show is sometimes very relevant and important.

Again, I believe we need to push students to be their best, but we need to be careful about overselling the "rush through the education" philosophy. Education is about learning, discovering, and developing...not about chasing grades and awards. I was reminded of this when I supervised the Debate programs for our school district during the 2013-2015 school years. One high school, Timber Creek High School, has a very successful Debate program. The teacher, Beth Eskin, is an inspiration to her students. The debate classes are full at TCHS because it is a cultural thing within their school community. A great majority of the upper-level incoming 9th graders all sign up for Debate. Many sign up because their older sisters and brothers were in Debate. I see the Debate programs at Apopka High School and Wekiva High School growing in a similar manner under Cathy Brown (AHS) and Kelli Mitchell (WHS - Kelli just left for Rome, Georgia, so there will be some big shoes to fill over there, but she let me know that an energetic teacher has already stepped up to the plate). One of the downsides of taking Debate is the fact that the level 3 and 4 Debate courses that a student would take during their junior and senior years of high school are honors level courses, so the super-smart students who are competing to become the Valedictorian have to make a choice--do they take an AP class to improve their GPA, or do they take Debate, something that will truly change their life, but it will hurt their GPA when compared to the other top-notch students in their grade level? If we were truly doing what was best for students in terms of their development, we would automatically enroll all students into Debate for all four years. The positive effects of taking Debate cannot be over-estimated. Many times, teachers of other subjects will rightly identify Debate students at TCHS just by the way the Debate students organize their thoughts and articulate their ideas. Every student should be exposed to such high level academic training. The other positive of such a requirement is students get to grapple with important issues of the day in a civil manner--something our society desperately needs. I especially like the fact that Debate students sometimes have to argue points of view they don't agree with, which forces them to seek understanding from another perspective. I would much rather see students "pushed" in their development in this manner rather than pushed to take courses that may be completely over their heads.

One top-notch Debate student at TCHS explained his decision to stay in Debate rather than run after GPA awards this way: He decided to not pursue being the Valedictorian because he saw how Debate helped him develop in the areas of critical thinking, research, listening, thinking on his feet, confidence, articulating his ideas, and organizing his thoughts. He said that Debate was preparing him for his future, while a GPA award may win him some immediate recognition, but it doesn't really do anything for what he wants to become in the long run. Besides, the Speech and Debate students travel to national tournaments throughout the year, and that's where the big Ivy League Universities do their recruiting...just like traditional universities recruit at sporting events on Friday nights. Speech and Debate students get accepted to the big universities long before other students because the colleges have seen these students in action as their academic skills and knowledge were on display while debating--they know what kind of student they are getting. So, I say all of this to make the point that school is about developing oneself...it's a journey...it's not a sprint. Sometimes, running after short-term goals and awards isn't the answer. We need to develop the total student for the long haul of life. I believe a commitment like this will not only develop highly successful students, but it will also create a better society in the process.

Traveling back to the other end of the spectrum, I also believe that we need to find venues where students can be successful within our high schools. I don't believe very many students are finding success at alternative schools where they do all of their work on a computer--the drop out rates at these schools isn't anything to brag about, so we can't say we are doing what is best for these students. What we are doing, to be totally honest and transparent, is simply getting a certain segment of our schools off of our books and out of our hair, which may make life easier for schools, but it does nothing for improving society and helping these students in the long run. Short run strategies may improve data and school scores, but it will not improve society. Our efforts to become a pipeline for these alternative schools is certainly nothing to brag about during an election year, and so it is left out of our data-driven discussions and celebrations. A better path would have been to find a career for these unmotivated students, get them skills that could lead to a high paying career in a technical field, and then send them out of high school with a plan for their lives. The school district has to invest in these kinds of options. We cannot only invest in technical programs at our new Orange Technical College. Students have to have at least a 2.0 GPA to travel off campus to participate in a program they may be interested in. If the program is at the local school, students can take a class with less than a 2.0 GPA, and many times, their success in a program they are interested in will help them succeed in all of their other classes...many times success is simply a result of the fact that they are in attendance at school more often. These technical programs are one of the best dropout prevention programs we can invest in...right up there with athletics and other extracurricular activities such as drama, choir, band, the debate team, the chess team, and other clubs and associations on campus. We, as a school district, have to choose to make this investment.

To finish up...One bit of statistics that we should ever keep before our eyes is the final result of all of our efforts in education. If all of our nation's students were reduced to a village of 100 students, 82 of those students would pass all of the required standardized tests, graduate, and receive a high school diploma. Of those 82 students who graduated, 65% (53 students) would go on to college. Of those 53 students who go on to college, 56% (30 students) would finish college in 6 years. Of those 30 students who finish college in 6 years, only 27% (8 students) end up working in a field related to what they received their degree in. For 8 students, the system works great. For the other 92 students, life is a journey. 53 students went to college, and many are saddled with debts they must repay--debts that cannot be erased by bankruptcy. If we aren't helping those who didn't go to college, and we are marginally helping those who did go to college, who are we really helping with the way we have education set up these days? The data does not indicate we are finding success. We need to make some data-transparent decisions regarding how we do business.

Now you know why I question people who say they make data-driven decisions. Data-driven isn't a bad thing, and pushing students to strive to be their best is hard to argue against, but we need to keep both of these two aspects of improvement in perspective. We need to broaden the discussion to other positive possibilities that could open the doors of success that we can only dream of reaching and realizing as things stand today.

I don't blame schools or school districts for playing the data game--if they don't learn to play the game, and play it well, they end up losing their jobs, which is a sad state of affairs for both our society, and the education world as a whole. We can do better...we must do better!


Featured Posts
Recent Posts
Archive
Search By Tags
No tags yet.
Follow Us
  • Facebook Basic Square
  • Twitter Basic Square
  • Google+ Basic Square

Political advertisement paid for and approved by Matthew J. Fitzpatrick, Non-Partisan, for Orange County School Board Chair

JOIN THE CONVERSATION: 

  • Facebook Clean Grey