Bulk Personal Datasets (BPDs)
The retention and examination of bulk personal datasets (BPDs) by the intelligence services is regulated by Part 7 of the IPA 2016. This defines a BPD as a set of information that includes personal data relating to a number of individuals, the nature of the dataset is such that the majority of the individuals are unlikely to be or to become of interest to the intelligence services, and that is retained electronically by an intelligence service and held for analysis in the exercise of its statutory functions.
	Part 7 sets out the safeguards that apply to BPDs. All datasets that meet the current definition of a BPD may only be retained and examined under a warrant that has been subject to prior judicial authorisation under the "double lock" authorisation process. BPD warrants are currently valid for six months.
	The "double lock" authorisation process requires warrants authorised by the Secretary of State to be approved by an independent JC before warrants can be issued.
	BPDs are used by the intelligence services in multiple different ways; for example, to provide ‘building block’ intelligence, such as names, dates, communication identifiers, details of travel, associates, etc. Traditionally, the critical value of a BPD is in the ability to make targeted queries of the data (for example, to identify a subject of interest ), cross-reference them with other BPDs and then overlay the results with other data from a variety of sources, (such as intelligence derived from other investigatory powers). This allows analysts to pull together an assessment on the possible meanings of the fragmentary intelligence that the intelligence services receive.
	Since IPA 2016 entered into force there has been a considerable growth in volume and types of data across all sectors of society globally, and at the same time the threat to the national security of the UK and its allies has diversified (as set out in the Integrated Review Refresh 2023 2 ). The information the intelligence services require to disrupt threats is increasingly fragmented amongst growing and varied data.
	The Home Secretary’s Statutory Review of the functioning of the Act stated that limitations within the IPA 2016 are inhibiting the intelligence services’ ability to maximise the benefits of digital transformation, and to ultimately protect national security. The intelligence services need to acquire increasing quantities of data, much of which is publicly available. It is anticipated that the data will improve analysis and in particular will enable the development of machine learning capabilities at the pace and scale the intelligence services need to identify and disrupt threats.
	As set out in Lord Anderson’s review, the IPA 2016 is restricting the intelligence services’ ability to make use of machine learning (ML) (including training to avoid biases) to support human lead analysis, and to manage increasing volumes of data and increase speed and quality of human decision making. It also restricts access to open resources such as telephone directories which can still be valuable for the more traditional uses of BPD.
	The training of ML models requires large quantities of open source or publicly available data that is representative of the type of data on which the model will be deployed, but which is voluminous enough to overcome or minimise any inherent biases.
	Unlike traditional uses for BPD, when training ML models the intelligence services do not examine the data to look for information on specific individuals featured in the data. Instead, BPDs are used for ML because they are representative examples of the structure or attributes of data the intelligence services are interested in. For example, the intelligence services may want to build a model to be able to identify weapons within images; the model will do this by learning from the training data features that make types of weaponry similar. Such models can be used to scan and triage images, before they are passed to human experts to assess. Developing models that can assist the intelligence services with growing volumes of data aims to make best use of resources in protecting national security.
	In his Independent Review of The Investigatory Powers Act 2016 3 Lord Anderson made the following recommendations, which are being taken forward via this legislation:
That IPA 2016 Part 7 should be amended to recognise a new category of BPDs in respect of which there is a low or no expectation of privacy, to which a distinct and less onerous set of safeguards should apply.
That IPA 2016 s213 be amended to provide that BPD warrants cease to have effect 12 months after they were issued, unless they have already been renewed or cancelled.
That IPA 2016 ss202, 206, 215, 219, and 220 (but not s210) be amended so as to provide explicitly that the functions with which they are concerned may be exercised by a Crown Servant on behalf of the head of an intelligence service.
	Building on these recommendations, this Act:
Amends safeguards for the retention and examination of BPDs where there is low or no reasonable expectation of privacy. This creates a new regime alongside the current Part 7. The intention of these changes is to enable the intelligence agencies to make more effective and efficient use of datasets in respect of which individuals have low or no expectation of privacy (such as online encyclopaedias and content from established news media).
Amends IPA 2016 s213 to allow for the extension of the duration of a BPD warrant from 6 to 12 months. Currently BPD warrants need to be renewed every 6 months. BPDs are often used to support long-term strategic intelligence activities rather than short-term tactical actions. The aim of introducing a longer warrant duration is to enable the value of the BPD to be more appropriately and accurately demonstrated.
Makes clear that the head of an intelligence service – the agency head – can delegate certain existing functions in relation to BPD warrants. This enables agency heads to delegate certain functions to an appropriate Crown Servant, whilst still being accountable for decisions that are taken on their behalf. The agency heads would still be required to personally carry out functions where risks are higher (such as under the existing duty in s210 to cease activity where a judicial commissioner refuses to sign off an urgent BPD warrant and the agency head must ensure the activity ceases).