Winter/Spring 2004
|
|
Winter/Spring 2004 |
 |
 By
Richard EkmanPrior to the mid-20th century, a person who wanted to clinch an argument needed only to reference a pertinent quotation from an authority. Politicians of the day cited Jefferson and Lincoln, while academics frequently cited Aristotle. More recently, those who wish to win arguments often cite numbers, not rhetoric—as if to suggest, for example, that because 58 percent of voters feel a certain way about an issue, the remaining 42 percent ought to feel the same way. We in higher education also have become more inclined to track a phenomenon through statistics, to argue that a past trend is a vector for the future. We have become adept at collecting some kinds of data, such as demographic projections, which have been particularly useful when combined with the actual patterns of student enrollment—by race and gender, full-time versus part-time, or family income level. Our records of students’ academic performance are also usually meticulous and complete. But analysts who try to use these records in combination with information about students’ nonacademic activities during their college years and beyond graduation have a more difficult time. The record-keeping systems on our campuses are rarely capable of sustaining ambitious, longitudinal studies. Increasingly, they will need to be. We cannot dismiss the crude measures of accountability that some in Congress wish to impose on colleges and universities, for there is no denying the sober intent with which Congress, many state governments, and an array of private organizations are questioning the effectiveness of colleges and universities and inventing methods to rate and rank them. Instead, we should welcome the better efforts to use data to illuminate the circumstances and achievements of colleges and universities. Such information can both enhance institutional decision-making and help to build a case for the effectiveness of private institutions. One diagnostic tool of particular utility is the National Survey of Student Engagement. Although not perfect, it is a helpful device to gauge the validity of our rhetoric about educational effectiveness. Because NSSE keeps its results confidential, individual colleges that score well will brag about their scores, while those that score badly remain silent. The decision to publicize one’s NSSE ranking can be a tricky choice: for a college to score, say, #10 out of 400 colleges sounds good unless the college’s main competitor is #9 (view story). Many other cogent uses of data to assess institutional effectiveness exist. For the stark truth about the connections between academic performance and participation in intercollegiate athletics at certain colleges and universities, the work done by William G. Bowen and his colleagues has been unflinching, revealing, and largely counterintuitive (view story). And a new diagnostic tool, RAND’s Collegiate Learning Assessment, is also a promising device for measuring how much cognitive growth takes place during the college years, on average, for students of a college or university. CIC is doing its part to help colleges and universities find and use meaningful quantitative and comparative measures of effectiveness. We believe in evidence-based decision-making, and hope to encourage policymakers to be guided by the strong record of independent higher education in comparison with other sectors. CIC continues to sponsor, in collaboration with the Association for Institutional Research, annual workshops for college and university personnel on the use of national databases for benchmarking. Surveys of alumni satisfaction are being carried out through a CIC contract with Hardwick~Day, a research and consulting firm, and we encourage all colleges to participate in such studies (view story). In addition, a draft “key indicators” tool, prepared by CIC through a contract with the Austen Group, was distributed at the Presidents Institute in San Diego, and widely praised as a useful benchmarking tool. The problems that were flagged by presidents in the draft (largely reflections of the flaws in the IPEDS data on which it was based) are now being addressed. Some presidents went so far as to say that these “key indicators,” when refined, will be one of the more important benefits of CIC membership. CIC members have usually been eager to learn from new data. However, for some presidents it may take real courage to use newly available data. One disturbing reaction to the “key indicators” draft from some CIC presidents was their fear that such a report, although intended to be a confidential diagnostic tool, might fall into the hands of an accrediting body or a potential funder that would use it in inappropriate ways. These complaints were just numerous enough to be a cause for concern at CIC, and I regret to say they were concentrated in one region of the country. It is worrisome that there is mistrust of a regional accreditor at a time when we most need to build trust in the importance of an evidence-based culture for higher education’s use of meaningful data. Two conclusions are suggested. The first is that we must continue to make the most of the emerging data opportunities—taking advantage of a new climate that encourages measurement, makes current data more readily available, and offers new, user-friendly ways for a college to work with these datasets. And the second conclusion is that the path to more and better use of data will probably continue to be mined with less-than-helpful efforts also made in the name of accountability—such as the overly simplistic U.S. News rankings or the punitive uses of data by an accreditor. It will not be easy to remain true to our ideals of candor in the use of new data on institutional effectiveness, but we must. Independent |
