You are hereevaluation
evaluation
Making Sense of Numbers: A Journey of Spreading the Analytics Culture at Tate
Elena Villaespesa, Tijana Tasich, Tate, United Kingdom
Abstract
Exploring the Relationship between Visitor Motivation and Engagement in Online Museum Audiences
Silvia Filippini Fantoni, Rob Stein, Gray Bowman, Indianapolis Museum of Art, USA
Abstract
In this paper, the authors will describe the rationale, methodology, and results of a series of studies that have been conducted with visitors to the Indianapolis Museum of Art website. The objective of the studies is to better understand people’s motivation for visiting the site and whether this motivation has an impact on the way they engage online. The hope is that these results will provide a reference dataset, and a replicable model for other museums that are interested in better understanding their online audience and in conducting similar studies for their own web efforts.
Delightfully Lost: A New Kind of Wayfinding at Kew
Natasha Waterson and Mike Saunders, Royal Botanic Gardens, Kew, United Kingdom
www.kew.org
Abstract
In October 2010, Kew Gardens commissioned an in-depth study of visitors’ motivations and information needs around its 300-acre site, with the express aim that it should guide the development of new mobile apps. The work involved over 1,500 visitor-tracking observations, 350 mini-interviews, 200 detailed exit interviews, and 85 fulfilment maps; and gave Kew an incredibly useful insight into its visitors’ wants, needs, and resulting behaviours.
It turns out that most Kew visitors have social, emotional, and spiritual, rather than intellectual, motivations during their time here. They do not come hoping to find out more, and they don’t want or need to know precisely where they are all the time. In fact, they love the sense of unguided exploration and the serendipitous discoveries they make at Kew—they want to become “delightfully lost.”
Levelling Up: Towards Best Practice in Evaluating Museum Games
Danny Birchall and Martha Henson Wellcome Trust; Alexandra Burch and Daniel Evans, Science Museum, UK; Kate Haley Goldman, National Center for Interactive Learning at the Space Science Center, USA
Abstract
Museums make games because games can provide compelling educational engagement with museum themes and content, and the market for games is enormous. Truly understanding whether games are achieving your goals requires evaluation. In this paper, we identify the kind of games that museums make and use case studies of our own casual games to look at the benefits and means of evaluation. Beginning by identifying different kinds of evaluation within the broad framework of formative and summative practices, we suggest ways to plan an evaluation strategy and set objectives for your game. We then look in detail at evaluation methods: paper and wireframe testing, play-testing, soft launching, Google Analytics, surveys, and analysing responses “in the wild.” While we draw on our own experience for examples of best practice, we recognize that this is an area in which everyone has a lot to learn, and we conclude by suggesting some tactics for sharing knowledge across the museums’ sector.
What do we really know about social media in museums?
With years of collective experience in social media engagement, many museums are stopping to ask themselves: what have we really learned about the audiences we are connecting with? Have we formed more meaningful relationships with our community and in the process, are we better serving the missions of our institutions?
Delivering mobile from the ground-up: exploring the approach
The challenges of delivering a meaningful ‘mobile’ experience for an institution are not always technology-related. Securing cross-departmental stakeholder investment and ensuring alignment with the institution’s core mission have been key for National Museums Scotland and National Museum Wales in developing a framework for their mobile programme.
Levelling up: Towards best practice in evaluating museum games
Games, and game-like experiences, are becoming increasingly important to museums seeking to engage new audiences and to provide deeper engagement with existing audiences. Demonstrating the value of commissioning and producing games as a core activity of museums is imperative.
Delightfully lost: a new kind of wayfinding at Kew
In October 2010, Kew Gardens commissioned an in-depth study of visitors’ motivations and information needs around its 300 acre site, with the express aim that it should guide the development of new mobile apps.
Museums in perspective: share and discuss daily data about museums and their audiences.
It is common practice among museums to publish quarterly or annual reports that include the museum attendance figures, online visits and more recently information about their social network activities. These figures are generally accepted as indicators of the effectiveness of museums activities both online and offline.
Using QR codes, mobile apps and wifi tracking data to understand visitor behaviour in exhibitions
In the last twelve months QR codes have reached a new level of hype within the museum community - increasing as their proliferation in street advertising in the USA and Europe has increased.
Let's Get Real
‘Let’s Get Real: How to evaluate online success?’ is a report published by Culture24 in September 2011 at their UK conference of the same name.
Rethinking Evaluation Metrics in Light of Flickr Commons
Paula Bray, and Sebastian Chan, Powerhouse Museum, Australi; Joseph Dalton, New York Public Library, USA; Dianne Dietrich, Cornell University Library, USA; Effie Kapsalis, Smithsonian Institution Archives, USA; Michelle Springer and Helena Zinkham, Library of Congress, USA
Abstract
In the past several years, cultural heritage institutions, including archives, libraries, and museums, have been placing their collections in Web spaces designed for collaboration and communication. Flickr Commons is one example of a highly visible space where cultural heritage institutions have partnered with a popular social networking site to provide greater discovery to, access of, and opportunities to interact with image collections on a large scale. It is important to understand how to measure the impact of these kinds of projects. Traditional metrics, including visit counts, tell only part of the story: much more nuanced information is often found in comments, notes, tags, and other information contributed by the user community. This paper will examine how several institutions on Flickr Commons - the Library of Congress, the Powerhouse Museum, the Smithsonian, New York Public Library, and Cornell University Library - are navigating the concept of evaluation in an emerging arena where compelling statistics are often qualitative, difficult to gather, and ever-changing.
Keywords: Flickr, metrics, evaluation, crowdsourcing, statistics, images
How to Evaluate Online Success? A New Piece of Action Research
Jane Finnis, Culture24, United Kingdom; Seb Chan, Powerhouse Museum, Australia; and Rachel Clements, Central Saint Martins, United Kingdom
Abstract
How to measure and define the success of cultural websites and online services is a problem common to all parts of the cultural sector – museums, archives, galleries, arts organizations, libraries and publishers. Worse, funding agencies and government departments lack the expertise to offer guidelines or set standards. The result is a confusing mixture of statistics and reporting that reveals little about genuine user satisfaction.
In mid-2010, a group of 24 organizations and agencies from across the cultural heritage and arts sector in the UK began collaborating on a piece of action research addressing the issues around 'evaluating online success'. Coordinated by Culture24 and working with Seb Chan, the groups of institutions have been developing a framework and best practice guide for measuring online projects both on organizational websites and those presented through social media channels.
Keywords: evaluation, metrics, evaluating cultural websites