British History Online: a new digital resource
Matthew Davies, Centre for Metropolitan History, Institute of Historical Research.
The purpose of this paper is to describe a pilot project currently underway here at the Institute of Historical Research, which will lead first to the creation of a unique and broad-ranging set of resources and tools for historians, and second to an assessment of the impact that digitizing such resources has on the way in which we go about the process of doing research. In 2002, funding was awarded by the Andrew W. Mellon Foundation for a one-year pilot study, and in December of that year Bruce Tate was appointed as project manager. The outcome of the pilot study will be a costed and timetabled plan to address key issues regarding the influence of digitization upon scholarship in the humanities, using a variety of digitized historical resources to test the hypotheses. This plan will include a framework for the development of a British history online library, encompassing management and administrative issues, ICT support, intellectual property licensing and acquisition and the future financial sustainability of the library. The web site for this project, which has recently been granted the domain name www.british-history.ac.uk, has now gone live and over the next few months we are hoping to gather as much feedback as possible about its functionality, design and content. Registration is free, and we hope very much that as many people as possible will register so that we can monitor use of the site and act on the feedback that we receive from users.
The origins of the pilot study lie in the IHR's long-standing role in the promotion and dissemination of online resources, and in the activities of the research centres based in, or associated with, the Institute. The three research centres involved in this pilot project are the Victoria County History (VCH), the History of Parliament Trust (HoP) and the Centre for Metropolitan History (CMH). All three centres have created, or make use of in the course of their research, extensive and valuable historical resources of different kinds. They include 'born digital' resources, such as datasets compiled from manuscript records, as well as printed sources, maps and images. Together these resources constitute an extraordinary mass of information about a huge range of historical themes, people and places. This pilot project has arisen from a perception not just of the value of these resources in their own right, but also from a growing awareness of the enabling role that could be played by technology in opening them up to researchers in new and exciting ways.
In the first section of this paper, I would like to say a little bit about the partners in this project, and the present form and value to historians of the historical resources that they have either created or use in their research. For the purposes of the pilot project a benchmark study of research methods currently used in connection with these resources has been carried out within each research centre, which will be used in subsequent assessments of the effects of this digitization programme.
Victoria County History
The famous red volumes of the Victoria County History have been published since 1900 and form a unique, and growing, record of the history of the English counties, their parishes, villages and towns. To date some 230 volumes have been produced, covering just under half the towns and parishes of England. Although the VCH is a very valuable national resource, it is quite difficult to use. Its indexes offer an efficient means of searching hard copy for a wide range of subjects across a group of parishes (i.e., those that constitute a volume), but the structure and form of the histories means that searching across the VCH is very time-consuming. This creates barriers to historians wanting to look at issues across regions or on a national scale. There is also a great deal of material hidden behind or omitted from the old indexes because it relates to subjects not thought to be of interest or significance when those indexes were compiled. The VCH has had it in mind for some time now to make the published volumes available electronically, marked up using XML, and to this end a pilot study was undertaken a few years ago, with funding from the Heritage Lottery Fund, which resulted in draft text from Oxfordshire and Darlington.
The History of Parliament
The Journals of the house of commons and the house of lords are one of the core sources for British history. For both Houses, the Journals exist in a more or less continuous series from the sixteenth century to the present day: for the Lords from 1509, and for the Commons from 1547. They were printed in the eighteenth and early nineteenth century from originals, many of which are now lost. They are often the only reliable record of what was done in parliament, both in terms of political debate and the mass of legislation - social, economic and religious - that parliament considered. Moreover, incorporated into the Journals is a vast number of separate documents of enormous historical value: records of trials and evidence in divorce cases; petitions; military reports etc. The Journals are, however, an under-used source. In part this is because they are difficult to search, and in part because of the sheer scale of their contents. Like the VCH, the History of Parliament is hoping to make the Journals into a more easily useable and searchable resource and, before this pilot project, had already undertaken its own experiment in retro-conversion using the 1,000 or so pages of volume one of the Commons Journals, covering the period 1547 to 1629. The larger scale digitization of the Journals would, furthermore, provide a basic source to which other resources could be linked, such as other online collections of parliamentary records, and the History's own biographical and constituency studies.
Centre for Metropolitan History
In the course of its research programmes the Centre has generated large quantities of data, much of it in electronic form, relating to the history of London from the eleventh to the twentieth century. The data sets contain, in all, more than 200,000 records, providing information on a wide range of subjects including trading networks, poverty, epidemic disease and merchants in London. They include the Cheapside Gazetteer - a detailed series of property histories relating to five parishes in London's premier shopping and marketing district from the twelfth to the late seventeenth century. Other examples include taxation records of various kinds compiled for the 'Metropolitan London in the 1690s' project, and data taken from nineteenth-century business directories. Currently, however, most of the datasets are only available to CMH staff and to visitors to the Centre. Recently the CMH has begun to explore the benefits of putting these datasets online, in conjunction with digitized printed sources, in order to facilitate data linkage and questioning in a way that would significantly benefit scholars working on a large number of themes in metropolitan history. Themes of particular importance include the reconstruction of individual careers, family and social structures, neighbourhood and parish reconstruction, the built environment and the overlaps and intersections between different 'groups' in London. Like the Commons Journals, data relating to these themes could be productively linked to other kinds of information, such as digitized biographical and topographical works that are currently out of print or unpublished.
The scope of the pilot study
The pilot study has been devised as a result of an awareness not just of the importance of these individual types of sources for historians in the fields of local, metropolitan and political history, but also of the exciting possibilities that are raised by the overlaps and interconnections between them. The digitization of the Commons Journals is in itself an important objective, and would be hugely beneficial for historians working on a range of themes, but the broader idea for British History Online is to put online a range of key texts, datasets and other sources that intersect and complement each other in important ways.
The aims of the pilot study have been as follows:
- to put online selected quantities of data from each research centre in order to establish protocols for digitization, mark-up, hierarchies and so on, and to develop some tools to search and model the data;
- to undertake benchmarking studies of research methods within each research centre, so that we have a record of how we use the resources currently;
- to consult with user groups about the development of British History Online (these have been extremely useful and have provided a great deal of information about the needs of historians when using online resources);
- to develop a framework and timetable for a much larger project, which would include the digitization of all the resources described above as part of a broad ranging study of the effects of digitization on research methods and strategies.
The most visible part of the pilot study at the moment is the website, where we have put online a range of resources. The form in which these are presented is gradually evolving as we take into account both the variations between types of sources, and the views of our user groups.
Online texts
I would like briefly now to describe the resources that you can find on the site and some of the ways they can be used, starting with the online texts contributed by the VCH and the History of Parliament.
VCH texts
The VCH has selected material available in two distinct forms. First, there is material that has been retro-converted from three published volumes of the Oxfordshire VCH, including part of the City of Oxford, the Borough of New Woodstock, and the parish of Great Tew. In other words, a sample of the kinds of places (cities, towns, parishes) that are typically covered by the VCH volumes. Second, we have unpublished text that can be modelled and adapted as it proceeds towards final publication: here the VCH has chosen text relating to the Parish of St. Clement Danes, and the City of Westminster, which provides an opportunity to make comparisons and links with the very different kind of material that the CMH is contributing for London.
History of Parliament text
A second group of retro-converted texts comprises two of the Commons Journals: volume one from 1547 to 1629, already digitized by the History; and volume three for 1642-44. Volume three has been chosen for several reasons. First, it covers a period currently being researched by the History and so will provide an important test-bed for how the Journals are likely to be used by researchers in this new format. Second, it contains extensive references to Oxford, mainly by virtue of parliament's executive role in these years, and the existence of the royalist court and headquarters at Oxford. This means that there is an opportunity to study the overlaps and connections with the VCH material. There are also many London references in the Journals, including people and places referred to in the CMH's datasets and texts. The Journals, such as that for 1643, can be browsed by month and then by day. Digitized images of the original text can also be viewed for any of the individual pages.
London history resources
One of the main concerns in the pilot is to study the interconnections between different kinds of resources, and how these could be exploited by putting them online. In the case of the CMH London data we felt that the best way of doing this was to focus on a very small area of the City: we chose one street, Ironmonger Lane in Cheapside - a street, however, that is exceptionally well-documented in the Cheapside Gazetteer, as well as in other datasets held by the CMH, and in printed sources. We have digitized approximately one-fifth of the Gazetteer, including the whole of the parish of St. Martin Pomary and its properties, and the complete index of 8,000 names. We have put online some taxation records for that part of Cheapside, including the subsidy of 1319 and the hearth taxes of 1662-3 and 1666, and some biographical and topographical sources, such as Henry Harben's Dictionary of London. There are also some digitized reconstruction plans, showing the changing property boundaries and street frontages from 1300 until after the Great Fire. For each map there is a palette of options, allowing you to zoom in on the map, or select particular properties to view their entries in the Gazetteer. The property numbers have also been linked, provisionally, to entries in the hearth tax returns, so that we can start to get a sense of who lived in each property at particular points in time. By clicking on a name in the hearth tax return, a search is carried out of the name index, enabling us to find out more information - where it exists - about particular people.
Modelling and searching the data
At the moment the main ways of retrieving information are by browsing, by using the indexes to people, and by using a search facility. We will be developing further indexes as we devise classifications and hierarchies for occupations, places and so on. One of the key findings from our user groups was the importance of a flexible search engine that could search across the resources, or just in a particular area of the site. By clicking on the advanced search option you will have access to the full range of searches, including Boolean and wildcard searches. A simple example is a search across the site for 'Cheapside'. This search comes up with a large number of references, among them an intriguing reference to the decoration of Cheapside Cross in London, and its referral to a committee for religion in parliament. The project manager has also built in a feature that allows users to save pages on a bookshelf for later consultation, and we have also been considering other options raised by our user groups, including the downloading of datasets such as taxation records, or creating an online calculations tool.
Another search facility has been put on the site as an example of the connections that can be made with external sites. The other search box you will find on many pages will allow you to search the Royal Historical Society's bibliographies, including London's Past Online, the CMH's new online bibliography of London history. By putting in a name or keyword you can bring up a new screen with a list of relevant books and articles, in other words allowing the researcher to assemble an even greater array of information about particular people, places or subjects.
As well as search tools, we are currently experimenting with mapping as a way to display and organize historical data. We have seen an example using the reconstruction maps for Cheapside, and there are variations on these for the parish of St. Clement Danes and for Woodstock. The next stage, however, is to develop a proper mapping resource, capable of being used within a GIS. We intend the resource to be based on the vector digitization of the earliest authoritative and detailed maps compiled by the Ordnance Survey between 1862 and 1872. In the last few weeks we have begun this process, using O/S sheets at a scale of 1:2,500 - these show parts of the Cities of London and Westminster, with the study areas on which the VCH and CMH are focusing for the pilot. The content of the map is being separated into layers corresponding to streets, buildings, parish and ward boundaries and so on, allowing the material to be combined with and/or linked to other data in various ways. We will then experiment with mapping some of the London data, including the 1665 hearth tax and the 1692-3 poll tax records, to represent visually such things as property-holding by inhabitants in different periods, occupational distributions, wealth and household size. Another source we will be using for the mapping stage is a series of business directories relating to the textile marketing area of the City in the nineteenth century. The VCH will use these maps in complementary ways, for instance as electronic, visual indexes to the VCH text, or in overlaying on them outlines of urban and rural estates to show trends in land-use and ownership.
Conclusion - the way ahead
So, where do we go from here? As outlined above, the aim of the pilot study is to establish a framework for the creation of a substantial online library, British History Online. This will emerge out of the report, to be written in the autumn of 2003, which will look at all aspects of the project and plot the way ahead. The framework for the larger project will allow for the large-scale digitization of resources of different kinds, whether they are further volumes of the VCH or Commons Journals, other digitized printed texts, or other datasets relating to the history of London. These resources will form the core of British History Online, but we see this not as a static collection of datasets, texts and so on, but as a 'living archive', so to speak, that will grow and develop as the subject itself evolves. This is essential if we are to assess the relationships between scholarship methods and the creation of digital libraries.
We also see this as an opportunity to create links with other well-established online resources, mirroring the IHR's role in the facilitation of historical inquiry and communication. We have, for instance, been in discussion with museums and heritage bodies and creators of other online resources, to think about the best ways of sharing data, or creating links that will enable scholars to pursue research in an imaginative and interdisciplinary way. This, we hope, will form part of a creative debate about research methods in the humanities, illuminating some of the opportunities and potential problems of collaborative research. This would have implications in other areas, for instance in the provision of training programmes for academic staff and graduate students in the uses of digital resources in research. Another aim is to incorporate new information formats, such as e-monographs, which the IHR believes to be an important step forward in the harnessing of digital technology to the benefit of humanities scholarship. Finally, this is very much a starting point. The input we have received from the academic community has been crucial, and we are very much looking forward to continuing this dialogue as we start to think about the future shape of British History Online.
