The authors designed a common evaluation framework for the Sealife Semantic Web Browsers including COHSE-NeLI based on the Conceptual Open Hypermedia System, CORESE-NeLI based on the CORESE engine and the GoPubMed/GoGene and tested the hypotheses(shorted for H) the authors proposed via camparing them to non-semantic control platform with participants recruited from the intervention systems’ real-world systems with substantial numbers of existing users.
The hypotheses made to test the key purposes (shorted for P) of the semantic web browsers are as follows:
P1: mobility and travel within the system
H1: The SWB reduces the time taken for users to find information or perform tasks.
H2: The SWB shortens the pathway taken to find information or perform tasks.
H3: Where semantic links are available, users will always follow them instead of nonsemantic links.
P2: User attitude and satisfaction
H4: Users find the SWB easier to use than the control platform.
H5: Where semantic links and ranking are available, users prefer them to non-semantic links and ranking.
H6: Use of the SWB is intuitive:
a) Users think the SWB helps them to find information or complete tasks.
b) Users intuitively understand how to use the SWB to find such information or complete tasks.
The evaluation results are showed in the table below.
The evaluation showed that users tended to prefer the system(GoPubMed) that had the most mature interface, but were able to use the semantic features of all systems regardless of the interface or types of semantic links presented. For one of the browsers (CORESE) less time was taken to complete tasks compared to the control system.