Using an RDF Data Pipeline to Implement Cross Collection Search

dhenry — Fri, 30 Sep 2011 15:56:18 +0000

MW2012

Published Paper:

Using an RDF Data Pipeline to Implement Cross-Collection Search

Keywords:

semantic indexing

Keywords:

solr

Keywords:

RDF

Keywords:

cross-discipline

Keywords:

Semantic Federation

Keywords:

collaboration, multi-institutional, data sharing, user submitted content, research, on-line community

Abstract:

The Missouri History Museum launched a first version cross-collection search in mid-2010. That implementation uses SOLR as a search engine with data from multiple domains (objects, archives, and photo collections) indexed to SOLR documents with various PHP scripts. Although some attempts were made to map data values to specific locations, dates, and subjects, most of the data is indexed as text fields. After user feedback and considering what would be possible with semantic web tools we identified a number of limitations to this approach: 1) text based facets are often ambiguous or vague; 2) users do not have the ability to further explore the context of search results; 3) data from different domains must be ‘watered down’ to conform to the search index; 4) data is not available as linked open data; and 5) we are missing an opportunity to contribute to the web of data through crowd sourcing.

To overcome these limitations the prototype of our next version is built firmly on sematic web technologies: RDF, SPARQL, and triple/quad stores. The prototype strives to meet the following requirements: 1) aggregate and index data from different domains and from multiple institutions; 2) expose a repository of linked open data; 3) index by specific data types (personal and corporate entities, geocoded locations, date ranges, and common vocabularies); 4) maintain the rich context of data from multiple domains and institutions; and 5) allow users to make contextual links between items, people, locations, dates, and subjects – thereby contributing to the web of data. At the heart of this approach is a semantic data pipeline that ‘ingests’ mixed data, converts it to RDF, transforms data to common data formats, aligns collection-specific fields to common fields, indexes to SOLR, and provides both a faceted search interface and a SPARQL endpoint for linked open data.

The prototype approach is not a single application; rather it is a proposed pipeline that uses many different tools that conform to semantic web standards such as RDF, RDFS, OWL, and SPARQL. Since they rely on web standards, tools may be swapped out for others that serve the same purpose without ‘re-building’ the pipeline. For example, there are many tools for converting between various data formats - called ‘RDFizers’, but our prototype uses D2RQ for converting relational databases to RDF; and modmarc2RDF for converting MARC records to RDF. There are many tools for converting between various RDF formats – our prototype uses the ARC2 library. Many tools exist for storing and querying RDF – we use Sesame. Finally, there are several tools available for transforming/mapping RDF data – we use DERI Pipes.

I will present the motivations and logic behind the proposed pipeline; discuss the pros and cons of various pipeline tools; and discuss the challenges and pitfalls we faced when building the prototype.

Type:

Mini-Workshop - teaches something or explores approaches

Mini-Workshop on Using an RDF Data Pipeline to Implement Cross Collection Search

Site:

http://collections.mohistory.org/search

Authors:

David Henry

Tagging

dbear — Mon, 29 Nov 2010 20:30:56 +0000

Type:

Session

Chair:

Timothy Hart

Programs:

Tag! You’re It! What value do folksonomies bring to the online museum collection?

Can Social Tagging Be a Tool to Reduce the Semantic Gap between Curators and Audiences? Making a Semantic Structure of Tags by Implementing Facetted Tagging System for Online Art Museums

Computational Linguistics in Museums: Applications for Cultural Datasets

Date & Time:

April 7, 2011 - 1:00pm - 2:30pm

Location:

Commonwealth A/B

tagging

social tagging

semantic indexing

computational linguistics

folksonomy

Required fields are marked with a red asterisk (*).

A confirmation of your registration will be sent to your email address. Please print it out and bring it to the conference.

You will not receive registration confirmation by regular mail.

If you do not receive your confirmation today, please contact us.

Archives & Museum Informatics EIN: 77-0708617; GST / BN 887978914

Description

Words, words, words. Whose words, why words, how words? Tags are just language but who assigns them, how and where, in in what relations to existing description is all critical to understanding, and ultimately to using, tags. In this session tagging discussion advances from art to science.

museumsandtheweb.com - semantic indexing

Using an RDF Data Pipeline to Implement Cross Collection Search

Tagging

Description