How do you measure importance?

Should importance be measured by popularity, by influence, or by something else? In the scientific world, the importance of an individual's contributions is often gauged by how frequently they are referenced by others. And the publishers of research journals know that revenue depends partly on their so-called impact factors, which measure how frequently their titles are cited. Sharon Begley (a fine journalist who covers science, and the business of science) recently wrote an insightful piece on some flaws in the process: Science Journals Artfully Try To Boost Their Rankings [Wall Street Journal, 5-Jun-2006; password required].

Among other things, she notes that a "concern is that impact factors, since they measure only how many times other scientists cite a paper, say nothing about whether journals publish studies that lead to something useful." That statement sums up some of the motivation behind my doctoral dissertation and the years I have devoted to my company, asula, since leaving Reed-Elsevier (a company that, rightly or wrongly, earns the distinction of being most-cited by people who like to talk about all that is wrong with the world of scientific publishing).

Citation analysis is useful -- I am not arguing against it. But I am arguing for a way to tell more of the whole story: For a way to look at how contributions tie directly to objectives and to results, so people can learn by seeing the various paths to potential outcomes. I am committed to providing a disruptive way to achieve more transparency, linking alleged "expert knowledge" with its influence on actual results.

Begley's article closes with a publisher's observation that "Ultimately, funding is allocated to scientists and topics perceived to be of the greatest importance." Sounds like what we need is a better way to determine importance.

Note: To learn more about citation analysis, see Citeseer and Thomson Scientific, two sources of journal citation information. For a glimpse of the future of scientific publishing, go to the Public Library of Science or Biomed Central.