We had another baby. :-)
Shai Brown Dueber was born last Monday, the 3rd, at a very moderate 7lbs 7.2oz (his brothers were 9lbs and 9.5lbs). Mother, baby, and older brothers are all doing well. Father is freakin’ tired.
Leave a Reply
Even better, even simpler multithreading with JRuby
July 1, 2011 at 12:22 pmCategory:Uncategorized
[Yes, another post about ruby code; I'll get back to library stuff soon.]
Quite a while ago, I released a little gem called threach (for “threaded #each”). It allows you to easily process a block with multiple threads.
- # Process a CSV file with three threads
- FIle.open('data.csv').threach(3, :each_line) {|line| send_to_db(line)}
Nice, right?
The problem is that I could never figure out a way to deal with a break or an Exception raised inside the block. The core problem is that once a thread trying to push/pop from a ruby SizedQueue is blocking, there’s no way (I could find) to tell it to wake up and see if there’s an error from another thread floating around that needs to be addressed.
So, I got into a pattern of running my code with each for a while, debugging, and eventually doing the production run under threach. Which is just dumb. Then I’d try to re-write threach to deal with this stuff using different approach (mutexes, lightweight events), quickly (or not so quickly) fail, give up, and start again.
So…let’s not worry MRI for the moment. I run all my big jobs under JRuby these days anyway, and there I can take advantage of Java’s blocking queues that have timeouts. When a queue operation times out, I can check to see if there’s been a break or an exception thrown in the meantime and behave appropriately.
The result is the gem jruby_threach. It works just like threach, except that, you know, it actually works the way I’d like it to.
- require 'jruby_threach'
- FIle.open('data.csv').threach(3, :each_line) {|line| send_to_db(line)}
Looks familiar, doesn’t it.
But you can also break out of the loop.
- myarray.threach(2) do |item|
- break if item_indicates_to_break(item)
- if item == :really_bad_value
- raise RuntimeError.new, "Something's really wrong", nil
- end
- process_item(item)
- end
Any exceptions that are rescued within the block are handled internally and don’t cause processing to stop. Any that are not handled within the block are noticed by threach, cause the processing to stop, and the re-raised so you can deal with them outside of threach
- reader = SpecializedFileReader.new(filename)
- begin
- reader.threach(2) do |item|
- process_item(item)
- end
- rescue SpecializedFileReaderError
- # deal with the fact that the reader failed
- rescue Exception
- # deal with the problem processing the item
- end
Dealing with the underlying Java data structures makes life a lot easier. To the point that I added an enhancement — threading production as well.
- # Use two threads to read lines from files, and another three threads
- # to process the data that comes out of those files.
- Dir.glob("*.csv").map{|f| File.open(f)}.mthreach(2,3) do |item|
- send_item_to_datbase(item)
- end
mthreach basically allows you to treat an array of Enumerables as a single logical entity, multithreading both the producer and consumer sides of the operation. There aren’t a whole lot of obvious use cases, but it can certainly come in handy.
You can also access the underlying class that aggregates multiple enumerables directly.
- require 'jruby_threach'
- me = Threach::MultiEnum.new(
- [enum1, enum2, enum3], # enumerables
- threads, # How many threads to use to
- :each_with_index, # the iterator to call on the enumerables
- size # size of the under-the-hood queue
- )
- # Note that like threach, calling #each against an MultiEnum actually
- # calls the iterator you sent in (in this case, #each_with_index)
- me.each {|item| process_item(item)}
3 Responses to “Even better, even simpler multithreading with JRuby”
Leave a Reply
Using SQLite3 from JRuby without ActiveRecord
Tags: jruby
May 26, 2011 at 2:20 pmCategory:Uncategorized
I spent way too long asking my friend, The Internet, how to get a normal DBI connection to SQLIte3 using JRuby. Apparently, everyone except me is using ActiveRecord and/or Rails and doesn’t want to just connect to the database.
But I do. Here’s how.
First, get the gems:
- gem install dbi
- gem install dbd-jdbc
- gem install jdbc-sqlite3
Then you’re ready to load it up into DBI.
- require 'rubygems' # if you're using 1.8 still
- require 'java'
- require 'dbi'
- require 'dbd/jdbc'
- require 'jdbc/sqlite3'
- databasefile = 'test.db'
- dbh = DBI.connect(
- "DBI:jdbc:sqlite:#{databasefile}", # connection string
- '', # no username for sqlite3
- '', # no password for sqlite3
- 'driver' => 'org.sqlite.JDBC') # need to set the driver
- # That's it. Everything below here is stock DBI
- dbh.do "create table squares (i integer, isquared integer)"
- ins = dbh.prepare("insert into squares values (?, ?)")
- (1..20).each do |i|
- ins.execute(i, i*i)
- end
3 Responses to “Using SQLite3 from JRuby without ActiveRecord”
-
Thanks! Only a question : i’m unable to retrieve values stored in the ‘squares’ table..
@dbh.execute("select * from squared";) do |row| puts row #or .. puts row['squares'] endand even with the prepare / execute statement but it always returns ‘nil’ or #
What i’m missing?
p.s. : I checked test.db with an external application and squares is correctly filled.
jruby and jdbc are up to date.
-
Sorry for bad formatting..and SQL string is : “select * from squares;”
-
At least with dbd-jdbc 0.1.4 it looks like the line:
require ‘dbd/jdbc’
should have the ‘J’ capitalized:
require ‘dbd/Jdbc’
Leave a Reply
How good is our relevancy ranking?
May 25, 2011 at 2:53 pmCategory:Uncategorized
For those of us that spend our days trying to tweak Mirlyn to make it better, one of the most important — and, in many ways, most opaque — questions is, “How good is our relevancy ranking?”
Research from the UMich Library’s Usability Group (pdf; 600k) points to the importance of relevancy ranking for both known-item searches and discovery, but mapping search terms to the “best” results involves crawling deep inside the searcher’s head to know what she’s looking for.
So, what can we do?
Record interaction as a way of showing interest
One possibility is to look at those records that are somehow “touched” by a user in such a way that we can log it. If a user bothers to interact with an individual record, we’ll assume the record is interesting to her in the context of the current search.
There are three links associated with an individual record that a user can click on from the search results:
- (62% of all record interactions) The title
- (28%) An external link (HathiTrust, Google Books, or one of our vendors)
- (10%) The “see holdings” link for those items that have multiple holdings
Our first issue arises quickly: only about a quarter of Mirlyn sessions contain any of these actions. For a full 75% of sessions, we have no data about which records users are paying attention to. They get a call number — or determine they have a failed search — and move on.
Where on the page do users interact with items?
We don’t know how users that interact with items differ from those that don’t. But for those that do, more than half of all record interactions are with the first record.
Here are the numbers for the first five records:
- First record: 54%
- Second record: 12%
- Third record: 6%
- Fouth record: 3.7%
- Fifth record: 2.5%
More than 75% of all record interactions are with the first four items on the first page of results.
What does it all mean?
Frustratingly, we don’t know. Several possibilities are obvious:
- we’re doing a good job with relevancy ranking
- people do mostly known-item searches
- people don’t bother looking past the first few results
- excellent general search engines (e.g., Google) have trained people to believe that the first result is always worth a closer look.
The interactions between these (and unknown other) factors are likely complex.
In the meantime, though, to the extent these data can be extended to the general case (not at all obvious), we’re not doing too bad of a job.
Leave a Reply
Ruby gem library_stdnums goes to version 1.0
Tags: librarian code, ruby
May 6, 2011 at 3:32 pmCategory:Uncategorized
I just released another (this time pretty good) version of my gem for normalizing/validating library standard numbers, library_stdnums (github source / docs).
The short version of the functions available:
- ISBN: get checkdigit, validate, convert isbn10 to/from isbn13, normalize (to 13-digit)
- ISSN: get checkdigit, validate, normalize
- LCCN: validate, normalize
Validation of LCCNs doesn’t involve a checkdigit; I basically just normalize whatever is sent in and then see if the result is syntactically valid.
My plan in my Copious Free Time is to do a Java version of these as well and then stick them into a new-style Solr v.3 filter so I (and, by extension, you, if you’re interested) can have Solr do normalization during both index and search time.
Leave a Reply
A short ruby diversion: cost of flow control under Ruby
Tags: ruby
May 3, 2011 at 3:54 pmCategory:Uncategorized
A couple days ago I decided to finally get back to working on threach to try to deal with problems it had — essentially, it didn’t deal well with non-local exits due to calls to break or even something simple like a NoMethodError.
[BTW, I think I managed it. As near as I can tell, threach version 0.4 won't deadlock anymore]
Along the way, while trying to figure out how threads affect the behavior of different non-local exits, I noticed that in some cases there was still work being done by one or more threads long after there was an exception raised.
I re-discovered something that a lot of people already know: raise/rescue under MRI is slow, and under JRuby can be unbearably slow. How slow?
Let’s look at four simple blocks that exercise four different block exit strategies: break, catch and throw, raise with the normal single (or zero) arguments, as well as the three-argument version of raise.
| Simple break | Catch/Throw |
|---|---|
range.each do |i|
break
end
|
catch(:benchmarking) do
range.each do |i|
throw(:benchmarking)
end
end
|
| Raise (1 arg) | Raise (3 args) |
begin
range.each do |i|
raise StandardError
end
rescue
# do nothing
end
|
begin
range.each do |i|
raise StandardError, :hi, nil
end
rescue
# do nothing
end
|
In each case, we immediately exit the block without doing any work; the idea is to measure how long it takes to break out for each case.
So....let's run them each 100K times and see what happens, shall we? Times are in seconds, averaged over two runs.
| Ruby 1.8 | Ruby 1.9 | JRuby | JRuby --1.9 | |
|---|---|---|---|---|
| break | 0.12 | 0.07 | 0.29 | 0.21 |
| catch/throw | 0.35 | 0.28 | 0.64 | 0.48 |
| raise (1 arg) | 1.78 | 2.10 | 26.60 | 22.06 |
| raise (3 arg) | 1.85 | 2.13 | 0.45 | 0.45 |
The first thing to note is that this is 100K iterations. Three of the strategies are fast enough that you'd have to work really, really hard to notice them. In terms of speed, raise (3 args), catch/throw, and break are fast enough that you shouldn't bother worrying about them (although you should choose the method that makes your code easy to understand).
The second things to note is Holy Camoli! JRuby is slow there!
This Jira ticket tells the tale: The creation of the backtrace is very, very expensive for JRuby. That nil at the end of the raise (3 args) call suppresses the creation of that backtrace, so the speed is fine.
Three things worth saying here:
- If you're using
raise/rescuefor flow control, you're already doing it wrong. Reserve exceptions for, well, exceptional conditions that are only going to be raised once or twice, not all the time. - If you're writing code that, for some ungodly reason, is planning on raising a crapload of exceptions, use the three-arg version. I'm looking at you, gem authors.
- If you're writing your code without worrying about how it will work under multiple threads, well, please don't do that. Everyone has multi-core systems these days, and it's silly to not be able to use them. Plus, counting on Matz to never move to a VM with real threads is a big gamble.
4 Responses to “A short ruby diversion: cost of flow control under Ruby”
-
For flow control in ruby, there’s actually a throw/catch architecture, which is an entirely different beast from raise/rescue. Nobody hardly ever uses them, throw/catch, I never see em, never used em myself either.
Note: raise/rescue DO correspond to JAVA’s throw/catch. ruby’s throw/catch is something different: It can only be used in ‘static scoped’ situations, basically where the catch is in a static code block that’s a parent of the throw. But if people are using raise/rescue for ‘flow control’ scenarios in places where throw/catch would work…. would be interesting to benchmark the performance of throw/catch. throw/catch at least is indeed actually intended for flow control.
Maybe nobody uses em cause they smell suspiciosuly like the dreaded ‘goto’, but that’s essentially what you’re doing with raise/rescue if you’re using em for flow control too, and apparently that doesn’t stop some people? Very curious what code you saw that was using raise/rescue like this, it’s certainly not a recommended thing to do by anyone (I don’t think?).
-
PS: Am I the only one that never uses those raise syntactic sugar shortcuts? I always actually create the Exception object myself:
raise StandardError.new
“raise StandardError” does the same thing, it’s just a shortcut. And:
raise StandardError, “message” ==== raise StandardError.new(“message”)
I don’t know the way to avoid backtrace generation when throwing an actually explicitly created Exception object, but there probably is one.
-
And briefly looking up the documentation on throw/catch, I’m wrong about the catch having to be statically scoped in a block above the ‘throw’ (the page I found in the online old ruby book actually specifically tells you this isn’t the case even though you might think it is, heh). But I’m still confused about where throw/catch can actually be used. It’s like the least used ruby language feature ever. But if lots of people are using raise/rescue for flow control, maybe throw/catch ought to be marketted better.
-
Another blog figures out the same thing, posted on reddit. You beat them to it! http://www.coffeepowered.net/2011/06/17/jruby-performance-exceptions-are-not-flow-control/
Leave a Reply
ISBN parenthetical notes: Bad MARC data #1
Tags: bad data
April 12, 2011 at 12:22 pmCategory:Uncategorized
Yesterday, I gave a brief overview of why free text is hard to deal with.
Today, I’m turning my attention to a concrete example that drives me absolutely batshit crazy: taking a perfectly good unique-id field (in this case, the ISBN in the 020) and appending stuff onto the end of it.
The point is not to mock anything. Mocking will, however, be included for free.
What’s supposed to be in the 020?
Well, for starters, an ISBN (10 or 13 digit, we’re not picky).
Let’s not worry, for the moment, about the actual ISBN and whether it’s valid or not.
Wait, no, let’s go ahead and worry about it. It’s an easy enough script to write, although it takes a while to run.
8,630,794 Total records
3,220,666 Total 020a's
6,498 020a's that don't obviously contain an ISBN
8,407 that look like an ISBN but fail checksum test:
... so 0.26% of the ISBNs have invalid checksums
So, not bad at all, especially considering some of those are known to be bad, but are transcribed dutifully from the actual (mis-)printed book.
A lot of the malformed data (anything from which I can’t seem to extract something that looks like an ISBN) is pricing data, and most of it appears in system numbers that are close enough to each other that I presume it was just a bad batch.
What’s goes after the ISBN in the 020?
I’m no cataloger, of course, but it looks to me like the answer is “Something about how the book is bound together, or the publisher, unless you want to put something else there, and then, really, go ahead, because it’s not like anyone is ever going to want to parse this out, all we need to do is print cards with it for god’s sake.”
No, I kid, I kid! The actual rules are in Library of Congress Rule Interpretation 1.8, which reads, in part:
For a hardbound resource, there is no attempt to use a consistent term other than to use one that conveys the condition intelligibly.
I think it’s important to read that a second time, because it succinctly conveys the culture in which these rules were devised.
- Don’t worry about consistency, because your only reader is human.
- Defer to the cataloger.
- Being complete is more important than being consistent.
- Base your notes on your subjective view of the actual, physical item you’re presumed to be holding in your hands.
Interestingly (to me, anyway), it looks like the OCLC once had a (now deprecated) $$b subfield for binding information. Apparently it didn’t catch on.
What did I find?
So, let’s pretend I’d like to be able to differentiate between paperback and hardbound books. Probably useful, yes?
I went ahead and took all parenthetical notes from any field in the 020, split them on colon (’cause that seems to be the way they roll) and did some basic normalization:
- Eliminate numbers (so ‘vol. 1′ and ‘vol. 2′ count as only one pattern)
- Lowercase everything
- Turn runs of spaces into a single space
- Trim leading/trailing spaces
- Remove any trailing punctuation
I found 1,506,729 parenthetical remarks in the 020 subfields of our catalog.
The top twenty most common entries using those normalizations are:
- 402537 pbk
- 387406 alk. paper
- 99260 v # (e.g., “v. 1″, “v. 22″, etc.)
- 82918 cloth
- 51125 hbk
- 42036 electronic bk
- 41360 acid-free paper
- 38792 hardcover
- 28913 set
- 20358 hardback
- 19160 ebook
- 16264 paper
- 15269 u.s
- 12770 hd.bd
- 11793 print
- 10625 lib. bdg
- 10520 hc
- 8772 est
- 7767 pb
- 7639 hard
The kicker? These are the top twenty of 13,374 unique parenthetical strings found in the 020 field. Many of them are publishers, or cities, or whatnot, but an awful lot of them are variations on “hardcover” and “paperback.”
For example, a quick search for anything that might be “hard” (regexp: /h[ar]{0,2}d/) got me started on a list. Here’s just the 90 examples from that list that start with ‘h’:
hard | hard adhesive | hard back | hard bd | hard book | hard bound | hard bound book | hard boundhard case | hard casehard copy | hard copy | hard copy set | hard cov | hard cover | hard covers | hard sewn | hard signed | hard-backhard-backcased | hard-bound | hard-cover | hard-cover acid-free | hardb | hard\cover | hardbach | hardback | hardback book | hardback cover | hardbackcased | hardbd | hardbk | hardbond | hardbook | hardboubd | hardbound | hardboundhardboundtion | hardc | hardcase | hardcopy | hardcopy publication | hardcov | hardcov er | hardcovcer | hardcove | hardcover | hardcover-alk. paper | hardcovercloth | hardcoverflexibound | hardcoverhardcoverwith cd | hardcoverr | hardcovers | hardcoversame | hardcoversame as above | hardcoverset | hardcovertion | hardcver | hardcvoer | hardcvr | harddback | harde | hardocover | hardover | hardpack | hardpaper | hardvocer | hardware | hd | hd bd | hd. bd | hd. bd. in slip case | hd. bd.in sl.cs | hd. bk | hd. cover | hd.bd | hd.bd. in box | hdb | hdbd | hdbk | hdbkb | hdbkhdbk | hdbnd | hdc | hdcvr | hdk | hdp | hdpk | hradback | hradcover | hrd | hrdbk | hrdcver | hrdcvr
And that’s after eliminating things like places of publication, strings like “with…”, “plus…”, “alk. paper”, etc.
“Yeah, but you have to understand that historically…”
Stop hiding behind that.
I understand that at one point in time it probably made sense (to someone at least) to do it this way. I can deal with that.
What I can’t accept is that as I type this there’s a cataloger doing this in this way. Today. April 2011. Some, what? maybe thirty years since computer-based OPACs became prevalent?
These sorts of problems were recognized ages ago and should have been dealt with. Add a subfield. Invent a controlled vocabulary. Don’t worry about the legacy data; it’s always going to suck.
But why are we still producing sucky data???
To sum up
The point is that there’s a better way to do this stuff. Lots and lots of better ways, in fact. Time I spend dealing with crappy data is time I don’t spend making relevancy raking better, or building a better command language search option for my librarians, or working on ways to get a decent “more like this”.
The need is both dire and urgent; the latter because sooner or later we’re going to have to go to a “two state solution” with traditional MARC21 for many of our records and whatever comes next (RDA?) for the newer stuff. And every day we wait, that first category grows, and the growth rate keeps increasing.
And then there’s serials. Don’t talk to me about serials.
6 Responses to “ISBN parenthetical notes: Bad MARC data #1”
-
Okay, so, what do you suggest we actually DO about this?
-
What we should do is first making clear what parts of cataloging produce useless junk (like Bill did) and second clean up and normalize the data. A typical reason for avoidable quality problems is a lack of feedback. If you do not instantly get feedback about illformed data when you start to create a record, you will unlikely change your cataloging practice. So third we should create better cataloging clients. As long as cataloging rules are only written as rules instead of implemented as code that given error messages, I doubt that we get better data.
-
Bill, ISBN is one of the examples I use in my talks when I get to the point of “text versus data”. I have grabbed a couple of examples, but you’ve done a full-blown study, and I’m here to thank you for it! I will point people to this post for more info.
Chris, as to what we do… given that any library system in existence today has algorithms to separate the ISBN from the rest of the subfield, we need to add a new subfield to MARC to hold the text and make that separate permanent. Actually, we need to have done that 20 years ago, and I almost feel like now it’s too late to make the change worthwhile since it looks like we’re on the verge of moving beyond MARC anyway.
-
Yet another example of where the UKMARC format was superior. In UKMARC the ISBN was in subfield “a”, qualifying remarks in subfield “c” and price in subfield “d”. There was even a subfield “b” with a code to indicate the type of ISBN (e.g. whether it was the ISBN for a set of volumes or an individual volume).
Sadly UKMARC was abandoned about ten years ago because it was just so unsatisfactory trying to convert from USMARC to UKMARC. It’s always easier to convert metadata into a format which is less expressive, so we all moved to USMARC instead.
-
[...] could look in the 020s for a hint of whether it’s hardcover or paperback (which is really hard. And maybe try to figure out if multiple volumes of a multi-volume work are all checked out and [...]
-
[...] Bill Dueber differentiating between bindings [...]
Leave a Reply
Why programmers hate free text in MARC records
Tags: bad data
April 11, 2011 at 3:40 pmCategory:Uncategorized
One of the frustrating things about dealing with MARC (nee AACR2) data is how much nonsense is stored in free text when a unique identifier in a well-defined place would have done a much better job.
A lot of people seem to not understand why.
This post, then, is for all the catalogers out there who constantly answer my questions with, “Well, it depends” and don’t understand why that’s a problem.
Description vs Findability
I’m surprised — and a little dismayed — by how often I talk to people in the library world who don’t understand the difference between description and findability. AACR2 is clearly designed for description; once you’ve found a record, it does a pretty good job telling a human being what she’s looking at. With respect to a person who’s already got a copy of the record in her (virtual) hand, strings of text and reasonable abbreviations are…well, often good enough, let’s say.
But much of AACR2 is a giant mountain of fail when it comes to supporting findability — the ability for a machine to slice and dice the data in ways that can be mapped onto searches and transformations. What those of us on the business end of the computer need are well-defined values stuck into well-defined places that represent well-defined relationships.
Free text stuck on the end of a field fails all three of those criteria.
Machine Reasoning vs. Machine Parsing
When many people look at something like RDF, their first reaction is, “Great Googally Moogally! Just tell me the language! I don’t want to follow a chain of reasoning that’s seventeen steps long just to figure out the damn thing is in English!!!”
Of course you don’t. And you don’t have to. Someone — hopefully someone smarter than me — needs to write a program to do it. And we can.
Following all that logic — deriving relationships, figuring out eventual values, determining how to convert between various forms — is what I’ll call (for simplicity’s sake) machine reasoning. And machine reasoning — for the purposes of this discussion, anyway — is a solved problem. I’m not saying it’s not hard, and I’m not saying it might not take gobs of hardware resources. But we, the collective of humanity, know how to do it.
On the other hand, machine parsing — looking at all that free text that is sprinkled throughout our records and trying to turn it into something that is susceptible to machine reasoning — is vehemently not a solved problem. Even if you ignore all the misspellings, we’re still stuck with one-off abbreviations, lack of ordering, gobs of “local practice,” and iffy punctuation.
And, come to think of it, you can’t ignore the misspellings, either.
The point is this: good data trumps everything else. If there’s good, solid, well-defined data in computable places, we can (given some time) do damn near anything with it. If there’s human-entered, free-text, parenthetical-remark-type data, we’re pretty much stuck.
Examples?
Jonathan Rochkind just did a great post looking at LC call numbers, and how, well, they might be in a few different places, and may or may not be valid LC call numbers, and so on and on and on and on.
And my next post (hopefully tomorrow) will be an analysis of the first freetext in MARC I ever tried to deal with — the parenthetical remarks in the 020 (ISBN) field. If that doesn’t keep you up all night, well, I don’t know what will.
6 Responses to “Why programmers hate free text in MARC records”
-
I don’t think they don’t understand — I mean, the catalogers I deal with around here are pretty clever. I think that YOUR concerns are not THEIR concerns and they are working within the constraints of a system that has encouraged, and in some ways even forced, then to get cute with parens and abbreviations because no one was going to change anything to support their need to differentiate.
The typos are of course only human, so I’m whatever on those. The punctuation, though? I cannot complain about enough. ;-)
-
You’ll note I demonstrated how okay I am with typos by inserting one. I did that on purpose, you know. No, really….
-
Programmers hate MARC because it’s the cool thing to do.
MARC records contain a lot more than the description. You don’t mention the subject metadata that most contain. This greatly enables retrieval (or findability). You do mention class numbers, which are also not part of the description and also help with retrieva.
Also, there’s no reason to play findability and description off against each other. Each fills a different need.
-
[...] Older » [...]
-
Just guesing here, since I’ve always been Colection Developmenet/Acquisitions, but haven’t those MARC records been pulled in from a lot of different sources, over many years, from the hands of many different catalogers? I remember something about the older a librar’s records are (the cataloging records), I mean the more hands that have worked on them, throught the years, a steady increase of mistakes or differences. Example: your search results featureing hard,hardback, etc, are possibly not even from the same library, originally. The fun of copy cataloging? Just a thought.
Leave a Reply
Corrected Code4Lib slides are up
February 15, 2011 at 5:49 pmCategory:Uncategorized
…at the same URL.
I was, to put it mildly, incredibly excited about code4lib this year because, for once, I thought I had something to say. And I did have something to say. And I said it. But it was wrong.
I presented a bunch of statistics drawn from nearly a year of Mirlyn logs. The most outlandish of my assertions, and the one that eventually turned out to be the most incorrect, was that some 45% of all our user sessions consist of only one action: a search.
Unfortunately, I’d missed a whole swath of things I should have excluded. I’d remembered robots and stuff coming in from our link resolver and so on. I hadn’t counted on having to fight my own stupidity.
In short: catalog.hathitrust.org and mirlyn.lib.umich.edu share a common code base, as well as a Solr backend. I was correctly excluding all the HathiTrust stuff from my stats except for simple searches. What I ended up with was a whole lotta sessions with nothing in them but that search. Luckily, I noticed waaaay too many people coming in via the HathiTrust site (which I know doesn’t have a link to Mirlyn) and did more digging.
The slides have been updated with correct numbers. Luckily, even though the adjustment was pretty extreme, I don’t think many of my conclusions are invalidated, especially given corroborating evidence from an extensive survey conducted by our usability team (PDF). They conclude, among other things, that known-item searching is prevalent and relevancy raking is important across task boundaries.
The basic stats from the powerpoint, for those who don’t want to read all my notes:
- 17% of all sessions have one action: a search
- In only 28% of all sessions does the user see the Record View
- 75% of all logged actions that target an individual record (see the full record view, look at extended holdings, etc.) happen with a record in the top 6 search results
- 7% of sessions involve a user adding a facet
- 2% of sessions involve a user exporting records
2 Responses to “Corrected Code4Lib slides are up”
-
[...] « Newer | [...]
-
Actually what this is is a really good lesson in the difficulty of getting valid results from usage statistics. “valid” meaning “actually answers the question you thought you were asking” in general.
Doesn’t mean we shouldn’t look at usage statistics of course. But it takes care and time to get good numbers — and then more care and time to make sure the numbers actually allow you to draw the conclusions you want to draw.
Not saying you didn’t do that here, but when my staff often asks me “Can’t we just get numbers to answer this question,” I will use this as an example of how you can often end up with numbers that don’t mean what you think they mean, and it takes non-trivial staff time to get and analyze the numbers to try to answer the questions you want to ask — so let’s be clear and intentional with the questions we’re asking, instead of just asking for ‘all the numbers’.
Leave a Reply
[RETRACTED] Code4Lib 2011 Lightning Talk Slides
February 9, 2011 at 6:26 pmCategory:Uncategorized
DANGER! I was trying to re-verify my numbers and found a glaring and hugely important mistake. I’ll make a new post with the details, but basically I was counting about 180k sessions (out of only 735k) that I should have been ignoring. Please ignore my basic stats until further notice. See the new numbers and corrected slides for more accurate data.
===============
I did a little Lightning Talk at Code4Lib 2011 and cleaned up (and heavily annotated) my slides for anyone interested in them.
The focus was on some basic stats about usage of our OPAC, Mirlyn, in calendar 2010.
I’ll be doing some posts and/or more rigorous writing on this stuff soon, but wanted to get these up in a timely fashion.
Bill Dueber Lightening Talk Slides from Code{4}Lib 2011. (powerpoint .ppt file, 1.2MB)
3 Responses to “[RETRACTED] Code4Lib 2011 Lightning Talk Slides”
-
[...] This post was mentioned on Twitter by Dan Chudnov and Jennyann, Bill Dueber. Bill Dueber said: Slides from my lightning talk about OPAC stats are up. http://bit.ly/dMvgPb #c4l11 [...]
-
Really enjoyed your slides. Love the long presentation notes. most slides I can’t tell what the speaker is trying to say but for yours no such problems. Just added Google analytics to the catalogue so this is very timely, you have far better statistics but still good figures to compare.
Just finished reading a year’s worth of your blog post. Really really nice blog, I can’t fully follow 100% of the details (I’m technically a reference librarian), but really enjoy most of the entries and the statistics of the catalogue you report.
-
I personally (speaking only for myself) think this would make a good short Code4Lib Journal article, and encourage you to submit one.
s/jquery_threach/jruby_threach/
Whoops! Thanks, and fixed.
[...] written a jruby_specific threach that takes advantage of better underlying java libraries called jruby_threach that is a much better option if you're running [...]