<--Word Circuits ---->Information about CyberMountain Colloquium and the corresponding MOO
Some notes for CyberMountain (1999) about my recently completed PhD Thesis
The title of my thesis is _Hypertext Versions of Journal Articles: Computer-aided linking and realistic human-based evaluation_. The thesis was accepted by the Computer Science department at the University of Western Ontario (in London, Ontario, Canada). I'm not sure when an online (Postscript) version will be available. I plan to make a website out of an earlier active voice draft sometime this summer. The official version is in the passive voice 'cause that's what the committee wanted.
My overall objective was to develop and evaluate ways of automatically incorporating hypertext links into pre-existing scholarly journal articles. In my thesis I describe a rule-based approach for making three types of links (structural, definition, and semantic). The rules were based on James Allan's work (see Hypertext 1996 Proceedings for example). Structural links are a way of making explicit some connections between parts of the text. Definition links connect the use of a term, defined elsewhere in the document, to that definition. Links that connect parts of text that discuss similar things are semantic links. I distinguish several types of semantic links.
I used two information retrieval (IR) systems (Cornell's SMART system and Bellcore's Latent Semantic Indexing) to select links based on the content of the articles. I conducted an experiment to compare the performance of the links forged using these two systems.
The effectiveness of the links (and the rules used to make them) was tested by people reading the hypertext versions for information under a time constraint. The notion of applying rules to generate links in such a way that the rules, and not only the final hypertext, could be evaluated has not previously received much attention in the hypertext community. The method used explicitly includes this vital concept.
A within-subjects experimental design was used. Each of the nineteen experimental participants read one version of each of three scholarly articles in a different hypertext form (one had only simple links, the others had definition links and semantic links selected using one of the IR systems). Subjects' preferences were also measured.
Subjects' ratings of the utility of the various links shows a significant preference for structural links over semantic links. Definition links were preferred to structural links, although the result was not significant. No difference between the links created using the two IR systems was detected.
However, there were significant differences in the times that readers spent on documents created using the various treatments. When they read in documents with only structural links readers were more likely to have read the whole article, and their satisfaction scores were inversely proportional to their comprehension score.
It seems likely that other factors are more important to readers than pure link quality. It must be noted that the readers had little prior experience with the system before the experiment and that they only had forty minutes to read each of the three articles. This somewhat artificial constraint simulated the condition where a reader had to quickly locate information that may or may not be present in an article. It would be interesting to see how readers perform with such a hypertext journal system over a longer time reading more than one article.
The method of evaluating hypertext versions of journal articles for use by researchers may be applied to other hypertext versions.
In my thesis I argue that links are idiosyncratic and that rather than trying to make the best possible links within discursive works we should instead support readers who make their own annotations. I do not directly address issues of links between documents, links in non-discursive works, or non-linked types of hypertext.