Building a research data management service for the London school of hygiene & tropical medicine

2.3.1. Determine research data management needs within the institution

Much of the research performed at the London School focuses upon public health and tropical medicine. However, this does not simplify the process of determining data management needs within the institution. The requirements of social scientists using mobile devices to perform surveys in the field differ significantly from researchers working in a lab environment. In the early stages of the project, a needs analysis was performed to identify areas where researchers needed help and determine the third party obligations that affected their approach to data management. This investigation was conducted using a combination of methods:

• Academic researchers: a web survey was conducted with academic staff to gain a better understanding of their research, data and perceived needs.

• Upper management: desk research was performed to review policies and procedures related to research operations, IT services and academic departments, as well as requirements of the RDWG.

• Research funders: a list of research funders that frequently support LSHTM projects was produced and desk research performed to determine institution and project-specific RDM requirements (Knight, 2012a).

• Regulatory obligations: desk research was performed of national/international obligations that affect research data, particularly that which involves human participants, such as the Data Protection Act.

Initially, it was planned that the Data Asset Framework (Jones et al., 2009) would be applied to produce three-four case studies on researchers’ projects, similar to those produced by the University of Edinburgh (Ekmekcioglu and Rice, 2009), Oxford (Martinez-Uribe, 2008) and Southampton (Gibbs, 2009). However, the Project Manager’s unfamiliarity with the institution at the time meant it was difficult to locate researchers willing to participate, while a central services re-organisation taking place meant that many professional support staff were unavailable.

The RDM Service had greater success with the web survey, conducted using the Bristol Online Survey tool[8] over a five-week-period (4 October-18 November 2012). This produced 117 responses, representing 16.25 per cent of academic staff. The RDM survey consisted of 15 questions (Knight, 2012b) drawn from the data asset framework and DRAMBORA (McHugh, 2007) toolkits. Rather than apply the project-by-project approach encouraged by these methodologies, questions were re-worded to examine common data management practices applied by the researcher across all of their current projects. This approach provided a high-level overview of a large number of research projects taking place at the time, but did occasionally produce anomalous responses. For example, a small number of respondents indicated they held data in eight different locations, without indicating if it was the same data being replicated across these locations, or data from different projects. The survey also provided insight into the environment in which health research is performed and the challenges encountered. It was recognised that data management practices are influenced by many stakeholders, including standards bodies (e.g. ICH Good Clinical Practice), research funders, journal publishers, federal agencies (e.g. US Food and Drug Administration) and governments (depending upon the location of data collection and partner institutions). Meeting these requirements is resource-intensive, particularly when conflicting obligations require the researcher to re-negotiate an agreement.

Figure 1 outlines key challenges that researchers encounter when creating, managing and sharing data within the school (Knight, 2013).

The research issues encountered by LSHTM researchers are comparable to those identified in RDM surveys performed at other institutions, such as the University of Nottingham (Parsons et al., 2013, p. 22), Royal Veterinary College (Harrison, 2013, p. 12) and the University of Northampton (Alexogiannopoulos et al., 2010, p. 28). Although differences in survey methodology make it difficult to make an exact comparison, topics such as writing data management plans, use of institutional storage systems, security processes and documentation standards are common across institutions. Similarly, the recognition that many LSHTM staff were unaware of institutional services available or could not access them when working in the field matched findings expressed by Alexogiannopoulos et al. (2010) that Northampton’s institutional storage was underutilized due to capacity concerns and difficulty in obtaining external access. It is only when researchers’ needs are examined in detail, as explored in Section 2.3.3, that differences in working practice begin to emerge.

The survey results identified several areas where data management practice should be improved, most notably a lack of procedures for managing data following project completion. In many cases, researchers would store the data in the personal or department area of the school’s network and move onto the next project, except in rare cases where a funder required them to deposit the data with a third party data service. To address this gap, staff and students training sessions were organised to address ethical and technical issues associated with post-project management. Meetings were also held with IT services and research ethics to discuss issues raised and consider how they could be addressed within the institution.

2.3.2. Ensure data management is considered from project outset

Data management is increasingly perceived as a key component of good research practice that should be considered early to ensure opportunities are identified, risks are mitigated and appropriate resource are allocated before work commences. In many cases, researchers already describe their data management processes, either within a funder-mandated data management plan or a domain-specific research protocol document[9]. However, there remains a small number of projects that are not covered by either requirement and, potentially do not consider data management until they undertake the research process itself. To address this gap, the RDM service sought to introduce a requirement in its RDM policy (principle 3 in Table I) that all research projects must create a DMP and submit it to the RDM Service for review. This approach is increasingly becoming standard practice across the academic sector, implemented by a growing number of research-intensive universities (Horton and DCC, 2014).

It was recognised from the outset that the introduction of an institution-wide DMP would be resource-intensive, both for research projects producing the plan and the RDM staff that would review it. Therefore, three requirements were set to minimise the work involved:

1. Prioritise key projects: the institution is responsible for managing data produced by projects where it is the named grant holder, data creator or data manager. These projects should be identified and steps taken to ensure their data is managed appropriately.

2. Avoid unnecessary duplication: projects that have produced a funder DMP or research protocol document should not be required to complete an institutional DMP.

3. Offer guidance, not judgment: the DMP should be easy to complete and offer suggestions on approaches that may be taken in the local context. Supporting information, example responses and multiple choice options should be provided wherever possible.

To comply with the three rules, a principal investigator would need only create a LSHTM data management plan if it was LSHTM-led, working with new data for which it was responsible and funded by an external body that does not have existing DMP requirements. Projects that had written a funder DMP or research protocol could fulfil requirements by forwarding an existing document. Projects that are analysing secondary data only, not working with data (e.g. those funded for the purpose of organising meetings and workshops), unfunded or performed as part of a consultancy are exempt.

A 15-question institutional DMP template was created[10], building upon the Digital Curation Centre’s DMPOnline generic template[11] and similar work performed by the University of Bath (Cope, 2013), which underwent testing during 2014. Initial feedback has been generally positive, indicating that the questions were easy to answer, requiring 15-20 minutes on average. However, several participants commented that the DMP forms are difficult to complete for large-scale, multi-site projects, proposing that a separate DMP form be tailored for these projects. There was also a general concern that the DMP would be critically evaluated and the Principal Investigator labelled “a bad data manager” if they were not able to respond.

To determine if the approach could be sustained with only one full-time RDM staff member, we sought to establish the number of LSHTM projects that might be eligible for the new DMP requirement in a given year. We began by analysing a list of proposals handled by the research operations team in 2013 – during which 627 applications were submitted (the largest number of bids submitted in recent years) – and classified them according to eligibility (LSHTM-led, submitted to funders with no DMP requirement, not a consultancy). It was not possible to determine the project objective, which may have indicated if it was undertaken to perform new research or co-ordinate workshop/meetings. However, the information was sufficient to establish that a maximum of 247 of the 627 applications (39 per cent) would be expected to complete a LSHTM DMP form at the submission stage if the requirement had been in place in 2013. Although it was considered feasible to review and comment upon this number of applications, it would require a significant amount of time, which would prevent other activities being performed. In addition, the dependency on a single person to review DMPs would create a bottleneck, preventing research bids being submitted if that person was on leave or unwell.

The recognition that a pre-award DMP would be unsustainable prompted investigation of a second approach – request eligible projects complete a DMP following funding confirmation. It was recognised that this may be too late to provide input into projects that had not allocated sufficient resources to data management. However, it would enable the RDM Service to focus only on the smaller number of proposals that are guaranteed funding – an approach also applied by the University of Hertfordshire[12]. By re-analysing the research operations data set, 197 projects were identified with a start date in 2013, of which only 97 met the eligibility criteria. This figure was considered to be easier to manage with the limited resources available to the RDM Service.

The proposal to mandate the creation of a data management plan as a post-award requirement proved to be essential for gaining senior management approval. Although the value of data in health research was well recognised, there was a reluctance to increase the Principal Investigator’s workload when they are already subject to many external requirements. By introducing the requirement following confirmation of funding, they would have more time to spend on discussing their data management approach with team members.

2.3.3. Supporting evolving data management needs

Finally, there was a need to provide tailored advice in response to researcher requests. This is a core activity for many RDM Services, but represents an unknown activity for those wishing to plan resource allocation. A fledgling RDM Service is unlikely have accurate figures on which to estimate demand and there is currently little information available on the number and type of support requests handled by comparable institutions.

In the early stages of service planning, we attempted to estimate potential demand in different scenarios, following the recommendations of the Keeping Research Data Safe project (Beagrie et al., 2010) and LIFE project (Ayris et al., 2008). For example, if an average of 300 research bids submitted each year have a DMP component, of which 20 per cent require three hours of support, it may be estimated that 180 hours will need to be spent reviewing research bids. Following service launch, we began to record details on each RDM support request submitted. Initially, the information was captured to inform the RDM Steering Group of progress. However, over time it offered insight into the faculties that were using the RDM Service (and those that were not) and areas where advice was most needed.

The number of support requests submitted to LSHTM’s RDM Service has grown steadily over the past two years. Following the RDM Service’s launch in November 2012, 13 queries were received and processed in November-December 2012, increasing to 88 queries for 2013, and 120 during 2014. This number can be easily processed by one person, in conjunction with other work. The number of support requests broadly match those reported by the University of Southampton, which indicate that 90 requests were submitted to their RDM Service e-mail account in the first year following their “soft launch” (White and Coles, 2014, slide 26). However, they are much smaller than the 158 RDM support requests processed over a three month period by the University of Manchester (Beard, 2014, slide 8).

Following recommendations by Jones et al. (2013), we encourage staff and students to submit questions via the RDM Service e-mail account, to provide continuity if and when staff leave. However, in most cases support requests are sent to the RDM Project Manager directly via e-mail. When asked the reason for this, researchers indicated they prefer to contact a named person, on the basis that they are likely to receive a quicker response. A small number of queries of support requests are communicated via telephone, scheduled drop-in sessions or simply visiting the RDM office for advice.

Figure 2 provides a breakdown of LSHTM support requests from each of LSHTM’s three research faculties (Epidemiology and Population Health (EPH), Infectious and Tropical Diseases and Public Health and Policy), as well central services (student registry, library, etc.), London International Development Centre and external researchers (researchers and students in other institutions).

An analysis of the number and source of support requests submitted each month can be used to provide insight into which departments are using an RDM Service and which are not. Staff and students in the EPH faculty show greater awareness of the RDM Service as a place for advice, having submitted 105 queries over a 26-month-period. However, less than half of contacts were from ITD (52 queries) or PHP (37 queries). On investigation, it was found that many researchers in these faculties consulted domain experts for advice, such as the LSHTM’s Quality and Governance Manager, or visited specific web sites, such as the UK Data Service. Although it is reassuring that researchers utilise existing resources where possible, it has led to a recognition that much of the guidance available through the RDM Service focuses upon broad needs and that further work could be done to produce resources tailored to specific subject domains.

There does not appear to be a correlation in the number of support requests submitted during particular months, beyond a recognition that fewer people get in touch at the end of term (July and December) when many staff are on leave. However, there is a noticeable increase in support requests during months where dissemination activities are organised (lunchtime seminars in January, February, April and October 2013, a drop-in session during June 2014, and half-workshop in November 2013-2014). External factors also influence the number of requests received during specific months – the majority of queries during March 2014 were prompted by growing awareness of the PLOS Data Policy[13], whereas many of the May 2014 requests focused upon a MRC funding call.

To identify broad themes or trends in the support requests submitted, the RDM Service began to assign a label to each query at the start of 2014. This proved to be difficult due to the wide-ranging and overlapping topics covered in many enquiries, but was helpful in identifying common needs.

Although topics covered in Figure 3 are likely to be encountered by many RDM/IT services across the academic sector, the underlying questions often reflected the specialised environment in which health research is performed. The majority of funder data management plans produced/reviewed by the RDM Service were prepared for submission to health-related funders, such as the Medical Research Council, Wellcome Trust, Cancer Research UK and Economic and Social Research Council; correspondence on the PLOS data sharing requirements are a result of its impact in the medical and health science field; and the large number of requests on data sharing agreements reflect the international environment in which health research is performed. Other requests, such as those related to data storage, security, encryption and sharing also often possessed a health-related element, such as consideration of the practicalities of working with personally identifiable information.

By identifying where support is most needed, an RDM Service can plan work to reduce the number of basic requests that will be submitted on a topic and the time needed to process them. For example, we produced a PLOS Data Policy summary guide (Knight, 2014b) and a worked example of a Wellcome Trust Data Management Plan (Knight, 2014c). In addition, the planned theme of a half-day workshop in November 2014 was changed to focus upon the challenges of sharing health data in compliance with journal and funder expectations. In the event that similar questions are raised at a later date, researchers can be directed to existing resources, reducing the processing time required to handle these queries.