museumsandtheweb.com - RDF

Using an RDF Data Pipeline to Implement Cross-Collection Search

dhenry — Mon, 26 Mar 2012 02:43:40 +0000

MW2012

David Henry and Eric Brown, Missouri History Museum, USA

http://www.mohistory.org

Abstract

This paper presents an approach to transforming data from many diverse sources in support of a semantic cross-collection search application. It describes the vision and goals for a semantic cross-collection search and examines the challenges of supporting search of that kind using very diverse data sources. The paper makes the case for supporting semantic cross-collection search using semantic web technologies and standards including Resource Descriptive Framework (RDF), SPARQL Protocol and RDF Query Language (SPARQL ), and an XML mapping language. The Missouri History Museum has developed a prototype method for transforming diverse data sources into a data repository and search index that can support a semantic cross-collection search. The method presented in this paper is a data pipeline that transforms diverse data into localized RDF; then transforms the localized RDF into more generalized RDF graphs using common vocabularies; and ultimately transforms generalized RDF graphs into a Solr search index to support a semantic cross-collection search. Limitations and challenges of this approach are detailed in the paper.

Program Item Reference:

Using an RDF Data Pipeline to Implement Cross Collection Search

dhenry — Fri, 30 Sep 2011 15:56:18 +0000

MW2012

Published Paper:

Using an RDF Data Pipeline to Implement Cross-Collection Search

Keywords:

semantic indexing

Keywords:

solr

Keywords:

RDF

Keywords:

cross-discipline

Keywords:

Semantic Federation

Keywords:

collaboration, multi-institutional, data sharing, user submitted content, research, on-line community

Abstract:

The Missouri History Museum launched a first version cross-collection search in mid-2010. That implementation uses SOLR as a search engine with data from multiple domains (objects, archives, and photo collections) indexed to SOLR documents with various PHP scripts. Although some attempts were made to map data values to specific locations, dates, and subjects, most of the data is indexed as text fields. After user feedback and considering what would be possible with semantic web tools we identified a number of limitations to this approach: 1) text based facets are often ambiguous or vague; 2) users do not have the ability to further explore the context of search results; 3) data from different domains must be ‘watered down’ to conform to the search index; 4) data is not available as linked open data; and 5) we are missing an opportunity to contribute to the web of data through crowd sourcing.

To overcome these limitations the prototype of our next version is built firmly on sematic web technologies: RDF, SPARQL, and triple/quad stores. The prototype strives to meet the following requirements: 1) aggregate and index data from different domains and from multiple institutions; 2) expose a repository of linked open data; 3) index by specific data types (personal and corporate entities, geocoded locations, date ranges, and common vocabularies); 4) maintain the rich context of data from multiple domains and institutions; and 5) allow users to make contextual links between items, people, locations, dates, and subjects – thereby contributing to the web of data. At the heart of this approach is a semantic data pipeline that ‘ingests’ mixed data, converts it to RDF, transforms data to common data formats, aligns collection-specific fields to common fields, indexes to SOLR, and provides both a faceted search interface and a SPARQL endpoint for linked open data.

The prototype approach is not a single application; rather it is a proposed pipeline that uses many different tools that conform to semantic web standards such as RDF, RDFS, OWL, and SPARQL. Since they rely on web standards, tools may be swapped out for others that serve the same purpose without ‘re-building’ the pipeline. For example, there are many tools for converting between various data formats - called ‘RDFizers’, but our prototype uses D2RQ for converting relational databases to RDF; and modmarc2RDF for converting MARC records to RDF. There are many tools for converting between various RDF formats – our prototype uses the ARC2 library. Many tools exist for storing and querying RDF – we use Sesame. Finally, there are several tools available for transforming/mapping RDF data – we use DERI Pipes.

I will present the motivations and logic behind the proposed pipeline; discuss the pros and cons of various pipeline tools; and discuss the challenges and pitfalls we faced when building the prototype.

Type:

Mini-Workshop - teaches something or explores approaches

Mini-Workshop on Using an RDF Data Pipeline to Implement Cross Collection Search

Site:

http://collections.mohistory.org/search

Authors:

David Henry

Building Linked Data For Cultural Information Resources In Japan

Tetsuro Kamura — Wed, 30 Mar 2011 16:29:59 +0000

MW2011

Tetsuro Kamura, The Graduate University for Advanced Studies; Hideaki Takeda, Ikki Ohmukai and Fumihiro Kato, The National Institute of Informatics; Toru Takahashi, ATR Media Information Science Laboratories; Hiroshi Ueda, ATR-Promotions.inc, JAPAN

http://lod.ac/

Abstract

Museum information in Japan is maintained distributedly and nonuniformly. This leads to difficulty in crossover searching for museum information. The LODAC (Linked Open Data for ACademia) project is building a prototype system (LODAC-Museum) to aggregate information across multiple sources. We identify and associate artists and works from different museum collections to provide integrated views for them. The key technology is Linked Data. All the aggregated data is transformed to the standard metadata schema and linked to each other via generated ID resources.

Keywords: Linked Data, Linked Open Data (LOD), Semantic Web, Metadata, RDF, Museum information

Program Item Reference:

Building LOD for cultural information resources in Japan

The 24 Hour Museum Tunable RSS News Feed

jtrant — Wed, 02 May 2007 00:57:39 +0000

Pratty, J. (2005). The 24 Hour Museum Tunable RSS News Feed. (Trant, Jennifer, David Bearman, Ed.).Museums and the Web 2005: Proceedings.