Introduction
Why is it so important to archive websites?
By nature, Web content is ephemeral: sites disappear continuously and are frequently updated, involving the disappearance of what is often very valuable online information.
This medium is pervasive in our society and certainly today one of its most important representations. From presidential elections to music festivals, every event is an opportunity to create, update or close a website, a Twitter, a Flickr or a Facebook profile; to communicate about it, to feed blogs, forums, social websites with opinion, points of view, issues… Concerning Museums and Galleries, Net'art involves a new perspective of archives and memory and new technological issues.
As with any other media, we believe this medium deserves a memory, and it is essential to preserve what has cultural, heritage and historical value.
Challenges come from the features and richness of the Web: dynamics, volatility, variety of formats and Internet users' contributions, all attributes increasingly used. Archiving the Web requires special attention in order to retain its value and ensure its fidelity to the original.
The Internet Memory Foundation (IMF)
The Internet Memory Foundation (formerly the European Archive Foundation) is a non-profit institution based in Amsterdam and Paris. Since 2004, it has actively supported the preservation of the Internet as a new medium. The Foundation has developed a wide range of collaboration all over the word, and especially in Europe, both with half-a-dozen cultural institutions and with about thirty research teams, to fulfill its mission.
Thus, the Foundation is archiving dozens of Terabytes of data per month, providing a shared Web archiving platform in order to help institutions to easily and quickly start collecting websites, including dynamic content and rich media, enabling navigation in past versions of a website, as if it were still live.
IMF is currently involved in various research projects with institutions such as the Max Planck Institute for Computer Science (Germany), Yahoo! Research or Leibniz University Hannover - L3S Research Center. It is developing several technologies to support the growth and use of Internet memory, such as a new crawler and architecture for Web scale crawling.
Our purpose in this paper is to present a large overview of Web archiving from IMF experience.
First, we present an overview of Web archiving, highlighting Europe. Then we continue with an outline of the State of the Art in this field. Finally, we raise the challenges we have to face, with a special focus on complex multimedia content, which is one of the main issues for Museums.
1. Inventory of Web archiving
International interest
Since the beginning of the Internet Archive, the first notable Web archiving initiative and the most ambitious (http://www.archive.org/web/web.php) in 1996, a number of international, national and institutional Web archiving programs have been launched. From major national libraries or archives to consortia and university departments, one can find a wide diversity of Web archiving projects.
Our aim is not to give an exhaustive description of every such program but to summarize and highlight common facts and issues.
From 12 members in 2003, the International Internet Preservation Consortium (http://netpreserve.org/) has increased to 36 members all over the world (4 in Asia, 24 in Europe, 8 in North America and 2 in Oceania). At the beginning, membership was limited to charter institutions, but now, IPPC is open to all kinds of cultural institutions, such as libraries, archives and museums. Goals of this group are:
- To enable the collection of a rich body of Internet content from around the world to be preserved in a way that it can be archived, secured and accessed over time,
- To foster the development and use of common tools, techniques and standards that enable the creation of international archives
- To encourage and support heritage institutions everywhere to address Internet archiving and preservation.
It is in the development of common tools and standards that IIPC has had its main achievements.
The growing interest in Web archiving is also noticeable through presentations and workshops in international conferences. Only last year, these conferences included:
- European Conference on Digital Archiving, ECA 2010: "Catching the Web - Mirroring information society through website archiving" by Austrian Parliamentary Administration (Günther Schefbeck, 2010)
- European Conference on Digital Archiving, ECA 2010: "Intervening early to ensure links persistence: How the National Archives' Web Continuity project has helped to redefine What to keep?" (Amanda Spencer, 2010)
- European Conference on Digital Archiving, ECA 2010: "Archivage du Web: perspective archivistique" by University of Montreal (Aïda Chebbi, 2010)
- Association of European Research Libraries, LIBER 2010: "Introducing Web archives as a new library service: the experience of the National Library of France" (Sara Aubry, 2010)
- International Conference of the Round Table on Archives, CITRA 2010: "Where have all the records gone? Challenges and Opportunities in Capturing websites" by Heritage Communications the Coca-Cola Company (Philip Mooney, 2010)
- International Federation of Television Archives, FIAT/IFTA 2010: "Workshop on Web archiving" by the National Audiovisual Institute of France (Claude Mussou, 2010)
European overview
Because the Internet Memory Foundation is established and more involved in Europe, we focus here on Web archiving projects in Europe.
In the framework of a research project named Living Web Archives, funded by the European Commission from 2008 to 2011 (http://liwa-project.eu/), Internet Memory Foundation carried out a survey on Web archiving in December 2010, sent to European and International institutions. We summarize here its main findings.
Which institutions do Web archiving?
This survey has been sent to more than 360 institutions (most of them are European heritage institutions):
- National and Regional Libraries (19%)
- University or specialized Libraries (22%)
- National and Regional Archives (11%)
- Audiovisual and/or Broadcasting Archives (20%)
- Institutional Archives, as Parliament Archive, European Institutions and International Institutions (18%)
- Documentation or Archive Department of Museum (10%)
Most of the respondents (73 institutions completed the survey) are National Libraries (25%) and Audiovisual Archives (25%), following by National and Regional Archives (15%).
Thanks to this survey, we discovered that more institutions than expected have a Web archiving program: 41 active institutions have answered and many of them are not in IIPC (26); 17 institutions answered they do not have a current Web archiving program but they plan to have one in the next 3 years. Combined with the fact that 90.3% of survey participants answered they would like to be regularly informed about Web archiving, we can affirm that Web archiving is an expanding topic.
| Do you have a current Web archiving program in your Institution? | Response Percent | Response Count |
|---|---|---|
| Yes, it is fully operational | 29,7 % | 22 |
| Yes, it is operational but still experimenting | 18,9 % | 14 |
| Yes, we are just starting a Web Archiving project | 6,8 % | 5 |
| No, we do not but we plan to do it in a short-term (1 or 2 years) | 12,2 % | 9 |
| No, we do not but we plan to do it in a mid-term (from 3 years) | 10,8 % | 8 |
| No, we do not because we do not have funding for this project | 5,4 % | 4 |
| No, we do not because it is not in our mandate | 1,4 % | 1 |
| No, we do not because an other institution is already in charge | 6,8 % | 5 |
| No, other reason | 8,1 % | 6 |
| answered question | 74 |
Table 1: Maturity of Web archiving program (IMF Survey, 2010)
It is also interesting to note that 42% of institutions which have no Web archives consider that it is or should be in the mandate of the National Library. It is also clear that these heritage institutions would like to be more involved. Only 15-20% of respondents are involved in the process, yet the interest goes up to 60% when asked if they would like to be more involved in the selection policy (preservation and access issues are coming just behind with 58% and 53%), even if the majority of institutions do not want to be involved in the whole process (around 80%). That is another sign that Web archiving can't be ignored any more.
Because we did not receive any answer from Museums, we are not able to say much about Museums' specificities or needs. Nevertheless, in the third part of this paper we highlight some specific issues we could expect for this field.
What do they archive?
A Web archiving selection policy may have to be formulated in the context of an existing organization-wide selection policy, or analogous selection policies for their types of resources. For example, in a country with a legal deposit on printed resources, the National Library would articulate its Web archiving policy in the same way.
One approach is to take the decision not to select, as Internet Archive (http://www.archive.org/web/web.php). The unselective approach is also possible within more limited contexts, as national domains (as in France or Sweden). So content of collections depends on the kind of institution and also legal limitations.
According to results of the survey, only national libraries (33% of them) operate global crawls on their national Top-Level Domain, but the large majority (93%) focuses on selective and thematic crawls.
On the other hand, we find audiovisual archives (75% of them), focus only on their websites (only 38% collect according to Thematic and Selective crawls). We can imagine that Museums would have the same selection policy: their own websites, with any associated social Web, and maybe some other selective websites as main galleries, specialized forums or blogs.
Concerning thematic selection, institutions can focus on Web domain (such as cern.ch), on subject (presidential elections), or creator (such as all governmental websites).
How do they manage their Web archiving program?
A number of different models have emerged for developing and implementing a Web archiving project. They can be categorized as follows:
- In-house: programs are entirely resourced and managed within the institution (about 70% of respondents are crawling in house). For example, the National Library of France, the Digital Archive of Chinese Studies, the British Library or the Library of Catalonia.
- Outsourcing: programs have all or some aspects of the work operated by a contractor (most of these respondents declare that they choose or would choose to collaborate with non-profit Foundations (of 91% of outsourcing respondents, 30% of them collaborate with Internet Archive and 62% with Internet Memory).
- Consortium: programs are implemented in a consortium of institutions, using some degree of shared infrastructure, research and development and management.
Concerning the staff involved and the budget, numbers really depend on the current stage of the project. Thus, 38% of fully operation programs count more than 5 Full-Time Equivalent (FTE)s, 50% of experimenting and 67% of just started projects count to 2 to 5 FTE's, and around 60% of planned projects count 1 or fewer FTE.
Concerning the budget allocated to the project, curves follow the same evolution: 41% of fully operational institutions declare spending more than €200,000/year; 50.5% of experimental institutions declare from €10,000 to €200,000/year, and more than 70% of planned project institutions declare not to have any budget for this program yet.
Which access to collections do they provide?
It depends on legal aspects: all institutions do not provide the same access to their collections for the user.
| Answers | Response Percent |
|---|---|
| Access is online for anyone | 41.0 % |
| Access is online with restrictions | 27.9 % |
| Access is on site for anyone | 18.0 % |
| Access is on site with restrictions | 21.3 % |
| We do not have access: contents are in a completely dark archive | 21.3 % |
Table 2: Access and Restriction to Web archives (IMF, 2010)
The other interesting aspect of access is the way chosen to navigate in collections.
Search
Searching is the most common method. The basic solution is to search for content on the basis of URL. Nevertheless this solution means that users need to know the exact URL. However, because users are more and more accustomed to Google, the most powerful and sophisticated option is provided by full-text searching across the entire collection or just a part of it, with the possibility to include, or not, type of content (.pdf,…).
Browse
Browsing Web archives means that the collection has been first categorized, classified, and that a hierarchy has been determined. These classifications can be helpful but offer a single view on the collection depending on decisions made by librarians and archivists. It is also possible to mix both solutions as Internet Memory did for the UK National Archives (http://collections.europarchive.org/tna/adv_search/).
After this overview of Web archiving, we now discuss the operational aspects of a Web archiving program.
2. Web archiving project planning
Web archiving projects follow a generic production scheme which includes several phases, from the selection to the hosting and the quality review of archived content:
Fig 1: Web Archiving production scheme
At the end of the scheme (quality review), some phases, such as capture, might start again, depending on the overall quality of captured content.
These practical phases of the workflow are presented below.
Selection
The selection phase can be divided in three distinct steps:
- Preparation
- Discovery
- Filtering
Preparation
The preparation is an important phase of a Web archiving project as machines follow human orders. The archive quality will therefore depend on the collection definition and policy.
During the preparation phase, one needs to define as precisely as possible its target, its collection policy and any tools, which might be used.
As with building any collection, it is important to list what will be part of the collection and what will be excluded. In the context of Web archives, this can be done under several aspects:
- The defined collection policy of the institution, its scope: What needs to be captured and preserved?
The scope will define the target content to be crawled: broad or focused collection, totally or partially for a website. In some cases, this will avoid duplication between institutions or Departments and will guarantee as much as possible the consistency of the collection.
- Frequency of capture.
The team should decide if a selected website or part of a website should be captured daily (for example, because of a political or social event), weekly, monthly, or even six-monthly or yearly (because the content of the website is not updated very often).
- Technical limitations:
These can be related to the tools used to capture content. Problems can, for instance, be expected when trying to use a crawler that is best suited for very focused crawls for a large snapshot of a whole national domain. They can also be related to the number of resources available for the project and/or these resources' profile, etc.
The preparation phase is therefore crucial as the project plan should meet all defined needs and take into account any predictable limitation.
Discovery
Once the targeted content is identified, it is important to find out how to discover and "extract" it from the Web.
Relevant entry points to the content you wish to capture can be discovered before the crawling phase (heterogeneous discovery) or while crawling (endogenous discovery).
By entry point, we mean a URL used as a starting point by the crawler tool to discover resources from the Web. To define the crawler behavior, a scope (as well as a number of technical settings) needs to be associated to the entry point. This will, for instance, force the crawler to follow only URLs that are part of the defined domain name. (e.g.: 123.com is the entry point, the scope states that everything discovered under 123.com should be captured.)
The heterogeneous discovery is very close to any collection building approach. The entry points are discovered manually through hubs, search engines or any relevant sources listing interesting references. If this process is well known and easier to handle, it is also time consuming and only useful for selective crawls. Still, the quality of the collection built will be easier to monitor.
The endogenous discovery relies on the crawler "tool" chosen for the project. It consists in providing a number of relevant entry points to the crawler and letting it discover related URLs. This method is based on the assumption that 1/3 of links are cross-sites links. Obviously, the URLs provided to start the discovery should cover as many aspects of the collection as possible. If not, some URLs might not be discovered because of the way sites link to each other on the Web. For instance, two competitors will not link to each other's website on the web.
Filtering
Entry points selected through the previous phases are provided to the crawler "tool" to start a new crawling job.
From there, two types of collections can be created, indexed and hosted, based on the crawling parameters:
Extensive collections start from the entry points provided and focus on horizontal completion. This means that the crawler is set to go to a limited level of depth. On the way, links to other domains found are used as new entry points. This method will allow the discovery of many URLs but will ensure deep coverage by domain.
Intensive collections start from the entry point provided and capture any URL which belongs to the related domain (or part of a domain, depending on crawling parameters). With this strategy, the crawler goes as deep as possible to capture a maximum of discovered resources within a domain.
Management tools and SaaS solutions
Once the crawler and collection policy are defined, Web archiving teams need to use a management tool.
Management tools can fulfill different purposes, from the management of entry points, collections (name, description, etc), to the management of crawling parameters and even to the automatic launch and monitoring of the crawl themselves. If technical parameters can be set from the management tool, the crawl can either be realized in-house, or be out-sourced with the help of an SaaS (Software as a Service) solution, for example.
This obviously impacts the project's resources. If crawls and technical aspects need to be monitored in-house, an engineering team will be necessary.
Hosting and access to collections are also subject to the choice of running a Web archiving project in-house or out-sourcing it. As for the management tools, access tools such as the Wayback Machine are open source, and maintaining servers to host content is feasible.
Again, at the project level, the choice has to be made to decide if all or some part of the workflow should be outsourced for economical or technical reasons.
Quality assurance
Most of the SaaS solutions do not offer any quality review of collections.
Once content is captured, it is archived and either hosted to allow access to content, or delivered to the institution in the standard archiving format (ISO 28500:WARC).
The quality review is obviously based on the institution's needs and its collection policy. It can be manual or "automated".
Quality control is a time-consuming task when done manually. It requires operators to be trained on what is a Web archive is, as well as tools used to collect content and their limits.
Based on their knowledge of the crawler behavior, operators test links methodically to discover missing resources which can then be added to the archive if decided by the project team. Indeed, adding resources to the archive after the crawl can be considered as an improvement of the archive quality, but also as an alteration of the archive. In the latter case, temporal incoherence can be experienced; for instance, when adding some content to an archived version of a page which was already updated on the live version.
Some tools can facilitate manual checking: tools such as browsers' plug-ins or proxies, for instance, by listing 404's when loading a page.
Some basic bug-tracking tool can also be used to compile information about collections at the entry point level. This may allow assessors to define areas of difficulties or technical limitations that might require development.
Automated quality assurance requires the use of tools such as URL testers, which can be coupled with other tools to facilitate missing resources recovery and archiving. Such tools are usually developed internally and based on users' experience and needs.
3. State of the Art
Web archiving requires tools to collect, preserve and access archived Web content. The International Internet Preservation Consortium has developed tools and methods to process Web archiving at each step of the process. Furthermore, other institutions such as the Internet Memory Foundation, but also companies and universities, develop their own tools.
This next part proposes an inventory of the tools useful for the Web archiving process: for selection, crawler, access, and preservation.
Selection
Most of the institutions which are involved in Web archiving create some focus collection based on specific contents: event, thematic… Management tools are developed to help curators or archivists to follow the process.
- Netarchivesuite (http://netarchive.dk/suite/) developed by the two national deposit libraries in Denmark, The Royal Library and The State and University Library, is an open source tool that allows planning, scheduling and operation of Web harvests for selective and broad crawl with a built-in bit preservation functionality (replication).
- The Web curator tool (http://webcurator.sourceforge.net) is an open-source workflow management application for selective Web archiving developed by the National Library of New Zealand and the British Library, initiated by the International Internet Preservation Consortium.
- Archive-it (http://www.archive-it.org/) is a subscription service by Internet Archive to build and preserve collections. It allows harvesting, cataloguing, managing and browsing archived collections.
- Archivethe.net, AtN (http://archivethe.net) is a shared Web archiving platform operated by the Internet Memory Foundation. AtN helps institutions to easily and quickly start collecting websites including dynamic content and rich media, enabling navigation in past versions of a website as if it were still live. By using a shared platform, institutions retain full control of their collection policy (sites selection, depth, gathering frequency, etc.), while benefiting from a dedicated and optimized crawling infrastructure. They also retain custody of collected material, with regular delivery of complete copies of collected material in the ISO 28500:2009 WARC format.
Crawls strategy
The collects are made with robots named crawlers. Collecting the Web is not an easy thing to do. Robots function in a similar way to crawlers for search engines. They must fit all the heterogeneity of architectures, (dynamic, social Web…), languages and formats (video, text, audio files…) and also interpret links. Links are the center of crawling problems as they are the key to discovery of content.
Several types of links can be found:
- Explicit links: source code is available and full path is explicitly stated
- Variable link: source code is available but uses variables to encode the path
- Opaque links: source code is not available.
- Different strategies are developed:
- Httrack or Heritrix (http://crawler.archive.org) is the Internet Archive's open-source, extensible, Web scale, archival-quality Web crawler project. It works as a parser, extracts links and archives the content on the fly.
- The other method consists in downloading the page as a browser, replaying the page, and downloading the pages and repeating the operation for all the pages of the website. This technology has been developed by Hanzo (http://www.hanzoarchives.com/)
To improve the crawl process, modules have been developed. In the context of the LiWA (Living Web Archives) European Project, new technologies have been developed to extend the current state of the art for capture, preservation, analysis, and enrichment services altogether, to improve fidelity, coherence, and interpretability of Web archives.
- SPAM Module: Web Spam (automatically generated Web content to cheat search engines) has become a real industry estimated to represent up to 20% of all Web. In the context of Web archiving, we can easily understand that archiving Spam is a big challenge in terms of costs (saving time during crawl process and hosting) and content compliance.
The SPAM module enables assessment of the crawled Web hosts. For each host, the module defines a "spamcity" of the host and decides whether or not the host has to be archived.
- Rich media capture: The Rich Media Capture module (RMC), developed in LiWA, is designed to enhance the capturing capabilities of the crawler, with regards to different multimedia content types. The current version of Heritrix is mainly based on the HTTP/HTTPS protocols, and it cannot treat other content transfer protocols widely used for the multimedia content, such as streaming.
- Temporal coherence: website may take hours to be crawled, during which changes can be made or pages can become unavailable. The module includes methods for proper dating of Web pages and provides coherent crawls with complete and correct temporal metadata, including time-aware reference sources such as the Wikipedia history.
Access
A Web archive is a copy of a website recorded by a crawler at a specific date and time. Each institution involved in Web archiving develops its own strategy. The main large public archive is available through the Internet Archive and Internet Memory Foundation. Some archives have only restricted access. This is the case for the French National Library, where archives are available on site only for accredited researchers, or in Austria. To browse the archive, users can research by URL or by full text.
Most institutions use the open source Wayback Machine developed by Internet Archive with the support of IIPC. Through a form a user submits an URL. The tool provides a table with all the archive dates for a given URL. By clicking on a date, the user can browse the archive. The URL archived is rewritten on the fly by the Wayback to ensure that navigation is really done on the archive and not on the live Web. Other access tools use a server-side rewriting method.
Currently, most implementation of full text research is based on Lucene (http://lucene.apache.org/).
As Web archiving has become more popular, user requirements have been taken into account in order to develop smart access. Thus, several user interfaces have been developed. The UI framework developed by LIWA provides a new timetable browser and an organizer. Other institutions like UKWAC (UK Web Archive) provide access to the archive by collection and using a time frame.
Another interesting development is initiatives to embed Web archives as part of the user experience. The Internet Memory Foundation has implemented with its partner The UK National Archives a redirection service that enables a user to be automatically redirected to the archive when the content is no longer available online.
Preservation
The issue of the preservation is linked to the issue of format and metadata. Web archives are recorded in a specific ISO format: WARC.
ISO 28500:2009 specifies the WARC file format. The WARC (Web ARChive) format specifies a method for combining multiple digital resources into an aggregate archival file together with related information. Resources are dated, identified by URIs, and preceded by simple text headers, using MIME entities. By convention, files of this format are named with the extension ".warc" and have the MIME type application/warc. The WARC file format is a revision and generalization of the ARC originally developed by the Internet Archive to store information blocks harvested by Web crawlers.
4. Metadata
In terms of preservation, Web collection consists of collected documents and their metadata and form together a network of content and description of content. Technical dependency is key for Web material as well as for digital objects in general.
Several levels of metadata have to be collected and preserved to ensure the reliability of the archive. Metadata from the selection process give information about the context of the crawl, which policy content was chosen, its original place, and its evaluation. Metadata about documents (size, format, checksum…) can be leveraged for preservation policy (migration, emulation and bit-level preservation). In addition to this, Metadata about the process gathers information about the communication with the server during the archiving process (giving important context to the documents preserved).
5. Challenges
Challenges come from the richness and variety of Internet content and its increasing dependency on new publishing technologies. We will focus here on the most relevant issues for institutions such as Museums and Galleries.
Video
Internet is a perfect media to offer good visibility to videos: from videos such as interviews of artists, or visitors' experiences, to videos as full-fledged artistic work, videos of performances and happenings etc. These videos are stored on museum websites and/or hosted on external platforms like YouTube. Thus, you will find the same video of "TateShots: Ai Weiwei, one-to-one" on the Tate Gallery website and also on their YouTube channel.
But capturing Web video is quite complicated because the technology used aims at avoiding direct access to the files by the users. Two main categories of issues have been highlighted.
- For websites that use standard HTTP protocol to deliver video content, the difficulty consists in the various techniques used to obfuscate links to video files (2 or 3 hops and redirects): YouTube is a representative example.
- Some websites use transport protocols other than HTTP, such as RTMP streaming protocol (http://www.swr.de).
The Internet Memory Foundation has developed some tools to overcome these problems, but it seems difficult to design general solutions for dealing with all the websites hosting video content. Moreover, the State of the Art for delivering Web video is changing fast, and Web archiving technologies will have to adapt to each particular case.
Social Web
The Social Web is also becoming a new challenge for preservation. Our information society is characterized by radical changes in information creation, communication and citizen involvement (e.g., there are now more social network items created than Google searches). Social media are becoming more and more pervasive in all areas of life. For example, Museums and Galleries are using Twitter and Facebook to inform their communities of new events.
On the one hand, this material is both ephemeral and highly contextualized, making it increasingly difficult for an archivist to decide what to preserve.
On the other hand, the social Web is based on dynamic websites and uses some specific technologies to publish content. New preservation strategies have to be created to enable the capture of this content in terms of selection and crawls. These issues are addressed on the research level by the European Commission-funded ARCOMEM project (Collect-All ARchives to COmmunity MEMories - http://internetmemory.org/en/index.php/projects/arcomem1).
Net'art
In the context of the Museum, Net'art preservation represents a challenge. Net'art covers a wide range of practices. In a global definition, Net'art gathers all creations based on the Internet as the medium when they cannot be experienced in any other way. In this framework, the artwork is as much the design of interactive devices that produce life forms online as interaction with the social Web. The Internet is just as invested as a workshop. How does one preserve such an immaterial artwork in the state of the art of the technology?
The work of Albertine Meunier, a French artist, questions in a critical or amusing way websites such as Google, Facebook or freebase. In 2006, she began "My Google Search History": the artist is publishing month after month, on a website, a comprehensive inventory of Google searches, questioning the retention of personal data by search engines. The artwork includes applications and results. Both aspects have to be considered for preservation. How to collect such materials?
An interesting experience designed by the British Library in 2009 illustrated this. "One & Other" is a live artwork by sculptor Antony Gormley. Over 100 days and nights, each of 2,400 participants representing every region of the UK spent an hour alone on an empty plinth in Trafalgar Square. All participants, or plinthers, were filmed, and the videos were brought together on the project website (http://www.oneandother.co.uk). The website received over 7 million visits during the project. Overly complex URL structures include numerous variables, marked by ampersands, equal signs, session or user IDs, as well as referral tracking.
The question of the preservation of this website was at the center of the project. In this context, the British Library has implemented a new solution to cover this event, dealing with archiving rich media and replaying it.
6. Conclusion
Web archiving has a great future ahead, but huge challenges. Since the Web was born, it has evolved in many ways, and the continuous development of the Web will require Web archiving actors to think and adapt at a fast pace and to develop scaling technologies.
The Internet Memory Foundation has, for many years, and thanks to collaboration with many partners and in several research projects, put its best effort into meeting these challenges. Its research areas are mainly the scalability and automation of process and the development of tools to use Web archives in a relevant way. Indeed, the final purpose is not only to be able to capture Web content, but also to preserve and share it.
7. References
Brown, A. (2006). Archiving websites. London: Facet Publishing.
Hockx-Yu Helen, L. Crawford, R. Coram & S. Johnson (2010). "Capturing and replaying streaming media in web archive - A British Library Case Study". International Conference on Preservation of Digital Objects (iPRES2010, Vienna). Consulted January 31, 2011. http://www.ifs.tuwien.ac.at/dp/ipres2010/index.html & http://www.ifs.tuwien.ac.at/dp/ipres2010/papers/hockxyu-44.pdf
Masanès, J., R. Pop & G. Vasile (2010). Archiving Web video. Vienna 2010: International Web Archiving Workshop. Consulted January 31, 2011. http://iwaw.europarchive.org/10/IWAW2010.pdf
Masanès, J. (2006). Web Archiving. Springer-Verlag Berlin Heidelberg.
Meunier, A. (2006). My Google Search History. Publication November 11, 2006. Consulted January 31, 2011. http://www.albertinemeunier.net/google_search_history/google_search_history.htm
Tate Gallery, TateShots: Ai Weiwei, one-to-one. Publication January 27, 2011. Consulted January 31, 2011. http://channel.tate.org.uk/#media:/media/766906069001&list:/channel/playlists/45927933001&context:/channel/playlists
Tate Gallery, YouTube channel. TateShots: Ai Weiwei, one-to-one. Publication January 28, 2011. Consulted January 31, 2011. http://www.youtube.com/watch?v=4sy1AFYxDmo
