Well, here the online environment can work with you rather than agsinst you, since you can make displays that show you the hierarchies radiating out in all kinds of directions.

Ideally, for “Great Britain — Social conditions — 19th century” you’d see not just that subject and its books, but related subjects and their books as well, displayed in a way that makes it easy for you both to find books of interest and to shift your focus based on what you find.

See this link for an example of how it works for “Great Britain — Social conditions — 19th century” in a collection of about 40,000 titles. The display shows 7 titles with that subject, and also shows a few others with more specialized subjects (social conditions in England during that time, or social conditions for women). And then it goes on to shift the focus outward a bit, looking at books on social conditions in England without the explicit time qualifier, and so on.

So a “subject” according to your analysis is simply the pre-coordinated, for example,

“Great Britain — Social conditions — 19th century.”

If you just make that a link and show all things with exactly the same heading, will you get others? Well, in my catalog, you’ll actually get a few hundred, yeah. (And I TRIED to find a good example that wouldn’t do that! But maybe I didn’t try hard enough). But let’s pretend not.

Okay, but how many will you get if you look for “Great Britain — Social conditions”, or “”Great Britain — Social conditions – ANY“? A lot lot more.

How about “Great Britain — Social conditions — 19th century — something else“? A lot more there too.

These subject headings were designed for a card catalog world where they’d all be laid out alphabetically, so the “wildcard” strings I suggest would neccesarily be right next to the original subject.

Our challenge is to figure out how to present these things in a rational way in the online environment instead. But it’s definitely not only linking to things with exactly the same pre-coordinated subject heading — if that often gets you very few hits other than your origin record, it’s because that’s not what LCSH was designed for.

I’m in the “what can we do with the data we have” camp more than the “how should our data be improved” camp, by and large. If we’re using subjects to promote discovery; having 7 million records with suboptimal inconsistent data is not surprising. Even if there are patterns such as the ones you explored off the top of your head, retrospective cataloging seems unlikely. (whispering) In fact, perhaps human cataloging isn’t really scalable. That said, I will take LCSH over call numbers, but since we have both …

Count other things such as length of book, dates, authors, etc. When you get this far compare the subject headings with the additional counts to see whether or not their are a relationships. Are books of one type of subject generally longer than others? Was this subject heading assigned more often during specific years? Are there common authors within subject headings? Are the books in question available via full text? Can you get those full text books and determine whether or not the books were cataloged “correctly” by doing text mining against the full text. Are there sets of “better” words that could be used to describe the books?

Fun with counting.