Irrational Metrics: Quanity Over Quality


Graduation rate is the least accurate measure of an institution’s success. Basing one’s evaluation on the count of bodies to pass through a school is equivalent to evaluating a baker on the basis of cookies created- the metrics just don’t make sense. As with the school, our baker will reject notions of quality in order to increase the number of items he or she creates. In situations where quantity is the measure of success, quality is a feature working against the common goal. Likewise, if we evaluate our schools (or any other institution) on the basis of quantity we disregard the only thing that matters: quality of service.

The popularity of measuring on the basis of “number served” likely comes from the ease of data collection. How many students graduated this year? How does this number compare to other years? Other schools? Other states? On the basis of simplicity the simple process of addition and comparison provides a nice collection of talking points. Yes, we do see data with value but when considering this information in relation to the initial goal or, dare we say, purpose of an institution there is nothing of value present.

Measuring on the basis of quality is unpopular because it is difficult to do. How do we measure the value of a degree? How can one consider the value of an education from one school in comparison to another? The common data point is income generated: After graduating from School X how does the income of Student X change? When comparing Student X and Y from School X and Y how do the incomes compare? Comparison is the common game but we are not limited to these dimensions. What do the students think? The best person to evaluate an education is the student who pursued it. Does Student X feel School X prepared him/her for the job acquired upon graduation? Questions like this establish the qualitative level of information to enliven evaluation.

One should not limit data collection to this qualitative level though. This data is the easiest to distort. The best studies extend to large selection of the population and over a long period of time. In order to properly evaluate an institution we must consider both quantitative and qualitative data from a majority of the population for at least thirty years. This is expensive, messy and unfavorable to politicians who aim to use educational statistics as rhetorical fluff.

If one aims to make drastic changes to the educational system one needs extensive data to prepare and justify alterations. It is easy to make accusations on the basis of “bad data.” Limited information yields limited perspectives. In order to make major changes our society must drastically expand institutional studies that take into consideration the inherent distortion and influence of all involved.

Advertisements
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: