Monday, November 20, 2006

BCS Rankings and the Computer

There is debate every year about the BCS system, and rightfully there should be, as it is a flawed system. However, having computers involved in the ranking process is not the flaw.

Let’s remember what it was like before computers were involved in the process. We had two sets of polls: the AP where the writers votes and the USA Today poll where the coaches voted. Every year there were complaints that the writers and the coaches were getting it wrong, they weren’t getting the right two teams identified. People complained that coaches were too biased for their own conferences and too self serving in their voting (both are legitimate concerns, whether or not they occur or not; perception is reality).

People complained that the sportswriters did not pay attention to all the games either, and they too had their own personal biases. There was near consensus that an objective third party had to be incorporated into the system.

And so the computers were brought into the mix. And of course, by the end of the first year, there was article upon article about how the computers had it all wrong. Everyone kept saying, “look at how bad the computers are. They have State University ranked #2 while the Sportstwriters and Coaches have them ranked #5.”

Think about that statement for a second, and think about why the computers were used in the first place. Wasn’t the consensus that the sportswriters and coaches had the polls wrong? And we’re using them as the litmus test to show that the computer selections are wrong? Isn’t the fact that the computer selection differs from the Sportwriters and Coaches the whole point? It should be a good thing.

That’s like saying your friend Jay says a car is ‘Blue’. You don’t believe Jay, so you ask your friend Joe. Joe says the car is ‘Red’, and then immediately concluding that Joe is wrong because Joe’s answer doesn’t equal Jay's. Hey, Joe’s answer better NOT equal Jay’s. You thought Jay’s was wrong; if Joe’s answer equaled Jay’s, then Joe’s answer would also be wrong.

That’s not saying Joe’s answer IS correct… it could also be wrong. But comparing it to Jay’s answer to come to that conclusion is erroneous.

Keep in mind there are two groups of people who have the loudest opinions when it comes to the BCS computer rankings: the coaches and the sportswriters. The sportwriters because they are the ones doing the writing, and coaches because they are the ones being asked the question.

Both groups have a vested interest in the computers being wrong, because it challenges their own opinion. Now just because the coaches and sportswriters have a conflicted point of view doesn’t mean they are wrong; but you do need to keep it in mind when you read article after article condemning the computer programs.

Remember that computers are only tools, and they only do what they are programmed to do. No more, no less. The computers aren’t wrong; they only do exactly what they were told to do. The people who wrote the programs may have been wrong, and in fact, probably are. I have no doubt that the computer rankings are somewhat flawed. They are a ‘black box’, proprietary to each computer ranking service, so we don’t know what is really being used to evaluate the teams. And evaluating a football team for pure statistical data is never going to be 100% correct.

However, at least the computers do look at every single football team, and try to rank them together. The computers throw out any personal bias and look objectively. Coaches and sportswriters, by their own human nature, are not able to do so. And since you aren’t going to use a playoff system, and millions of dollars are at stake, you really should have an objective criteria.

Remember, you can program the computers to do anything. I could write a simple computer program today that would rank all the college football teams based on an average of the AP and USA Today polls. That folks, would be a computer program, and it would give results that many of you would probably find comfortable. In part, because it is very transparent… you know exactly what is going on. And also because the results would be very familiar to what you already see in the polls, so it’s within your comfort zone.

Moving forward (assuming no playoff system… which I’ve addressed elsewhere), I think you have to keep the computers in the process, to balance off the biases and limitations of the human vote. But it should only be a component, not the overall deciding factor. And those programs should also become transparent to the public; let people see how the programs work. And keep on improving them… they’ll never be perfect (because the people writing them aren’t) but they can keep getting better and better.
Remember my opening remark. "Having computers involved in the ranking process is not the flaw". That's not saying the programs themselves aren't flawed. They are. But its better than not having them.

No comments: