Risk Models – Think Family Database

Date of request: 19 April 2024
Reference: 596-24

Request

Questions have been copied below, in bold. Our responses follow beneath each question.

Response

Following correspondence with Bristol City Council I have been informed that you hold information on the risk models in Insight Bristol’s Think Family Database (TFD). 

Under the Freedom of Information Act 2000 I request the following information for the 6 risk models of the TFD 

Risk of Sexual Exploitation Model

Child at Risk of Criminal Exploitation Model

the Risk of NEET (Not in Education Employment or Training) Model

The sub-models of the Risk of NEET model

Attainment

Attendance

Vulnerability

the source code – i.e. machine learning model files – used to calculate risk scores in the above models

The above models have all been built within IBM SPSS Modeler. This system creates models in node-based programming. The system does not produce a source code.

the source code files which produce the risk model files, for example by selecting and preparing data for training, training the models, and outputting the risk model

As above, the system does not produce source code.

a description of the above models including name, model type (e.g. decision tree, random forest, line of best fit, KNN etc.), coding language written in, validation method, performance matrix, and confusion matrix

For models 3 and 4 (the Risk of NEET (Not in Education Employment or Training) Model and The sub-models of the Risk of NEET model, this information is no longer held. The data was last modified in 2017 and has been deleted due to retention.

With reference to models 1 and 2, please find the information requested below:

Name	Risk of Sexual Exploitation Model	Child at Risk of Criminal Exploitation Model
Model Type	Ensemble of the below CHAID C5.0 Neural Network Logistic	Ensemble of the below C5.0 Logistic
Coding Language	N/A The models are node based	N/A The models are node based
Validation Method	Not recorded however Provide list of children to subject matter experts (SMEs) to review We split the model into 80/20 training test split	Not recorded however Provide list of children to subject matter experts (SMEs) to review We split the model into 80/20 training test split
Performance Matrix	No report/or output is created the information is only created during development and not stored anywhere.	No report/or output is created the information is only created during development and not stored anywhere.
Confusion Matrix	No report/or output is created the information is only created during development and not stored anywhere.	No report/or output is created the information is only created during development and not stored anywhere.

4) a table which lists 
a) all variables used in the above models, and  
b) the weights of each variable and/or category of variable in the calculations performed by the risk model i.e. how much it contributes to the overall scores the model outputs 

Name

Risk of Sexual Exploitation Model 

Child at Risk of Criminal Exploitation Model 

Variables

The below list of variables are all imported at the beginning of the model process but not all are used. There is no output created for what is used as it is done as part of the process.

the process automatically keeps variables with predictive power to create a model. No report/or output of the variables is saved or created other then for viewing in the development stage so it not held

Victim_Burglary_00M_03M
Victim_Theft_00M_03M
Victim_Violence_Against_The_Person_00M_03M
Victim_Vehicle_Offences_00M_03M
Victim_Arson_and_Criminal_Damage_00M_03M
Victim_Public_Order_Offences_00M_03M
Victim_Possession_of_Weapons_00M_03M
Victim_Drug_Offences_00M_03M
Victim_Sexual_Offences_00M_03M
Victim_Miscellaneous_Crimes_Against_Society_00M_03M
Victim_Robbery_00M_03M
Victim_Burglary_00M_06M
Victim_Theft_00M_06M
Victim_Violence_Against_The_Person_00M_06M
Victim_Vehicle_Offences_00M_06M
Victim_Arson_and_Criminal_Damage_00M_06M
Victim_Public_Order_Offences_00M_06M
Victim_Possession_of_Weapons_00M_06M
Victim_Drug_Offences_00M_06M
Victim_Sexual_Offences_00M_06M
Victim_Miscellaneous_Crimes_Against_Society_00M_06M
Victim_Robbery_00M_06M
Victim_Sexual_Offences_00M_12M
Victim_Arson_and_Criminal_Damage_00M_12M
Victim_Possession_of_Weapons_00M_12M
Victim_Violence_Against_The_Person_00M_12M
Victim_Theft_00M_12M
Victim_Miscellaneous_Crimes_Against_Society_00M_12M
Victim_Public_Order_Offences_00M_12M
Victim_Drug_Offences_00M_12M
Victim_Robbery_00M_12M
Victim_Vehicle_Offences_00M_12M
Victim_Burglary_00M_12M
Victim_Violence_Against_The_Person_00M_24M
Victim_Theft_00M_24M
Victim_Arson_and_Criminal_Damage_00M_24M
Victim_Vehicle_Offences_00M_24M
Victim_Burglary_00M_24M
Victim_Public_Order_Offences_00M_24M
Victim_Miscellaneous_Crimes_Against_Society_00M_24M
Victim_Drug_Offences_00M_24M
Victim_Sexual_Offences_00M_24M
Victim_Possession_of_Weapons_00M_24M
Victim_Robbery_00M_24M
Victim_Violence_Against_The_Person_00M_36M
Victim_Theft_00M_36M
Victim_Arson_and_Criminal_Damage_00M_36M
Victim_Vehicle_Offences_00M_36M
Victim_Burglary_00M_36M
Victim_Public_Order_Offences_00M_36M
Victim_Miscellaneous_Crimes_Against_Society_00M_36M
Victim_Drug_Offences_00M_36M
Victim_Sexual_Offences_00M_36M
Victim_Possession_of_Weapons_00M_36M
Victim_Robbery_00M_36M
Victim_Violence_Against_The_Person_00M_60M
Victim_Theft_00M_60M
Victim_Arson_and_Criminal_Damage_00M_60M
Victim_Vehicle_Offences_00M_60M
Victim_Burglary_00M_60M
Victim_Public_Order_Offences_00M_60M
Victim_Miscellaneous_Crimes_Against_Society_00M_60M
Victim_Drug_Offences_00M_60M
Victim_Sexual_Offences_00M_60M
Victim_Possession_of_Weapons_00M_60M
Victim_Robbery_00M_60M
Victim_Violence_Against_The_Person_03M_06M
Victim_Public_Order_Offences_03M_06M
Victim_Arson_and_Criminal_Damage_03M_06M
Victim_Miscellaneous_Crimes_Against_Society_03M_06M
Victim_Sexual_Offences_03M_06M
Victim_Theft_03M_06M
Victim_Drug_Offences_03M_06M
Victim_Possession_of_Weapons_03M_06M
Victim_Burglary_03M_06M
Victim_Vehicle_Offences_03M_06M
Victim_Robbery_03M_06M
Victim_Sexual_Offences_06M_12M
Victim_Burglary_06M_12M
Victim_Arson_and_Criminal_Damage_06M_12M
Victim_Possession_of_Weapons_06M_12M
Victim_Violence_Against_The_Person_06M_12M
Victim_Theft_06M_12M
Victim_Drug_Offences_06M_12M
Victim_Public_Order_Offences_06M_12M
Victim_Robbery_06M_12M
Victim_Miscellaneous_Crimes_Against_Society_06M_12M
Victim_Vehicle_Offences_06M_12M
Victim_Violence_Against_The_Person_12M_24M
Victim_Theft_12M_24M
Victim_Arson_and_Criminal_Damage_12M_24M
Victim_Vehicle_Offences_12M_24M
Victim_Burglary_12M_24M
Victim_Public_Order_Offences_12M_24M
Victim_Miscellaneous_Crimes_Against_Society_12M_24M
Victim_Drug_Offences_12M_24M
Victim_Sexual_Offences_12M_24M
Victim_Possession_of_Weapons_12M_24M
Victim_Robbery_12M_24M
Victim_Violence_Against_The_Person_24M_36M
Victim_Drug_Offences_24M_36M
Victim_Sexual_Offences_24M_36M
Victim_Arson_and_Criminal_Damage_24M_36M
Victim_Public_Order_Offences_24M_36M
Victim_Miscellaneous_Crimes_Against_Society_24M_36M
Victim_Theft_24M_36M
Victim_Robbery_24M_36M
Victim_Burglary_24M_36M
Victim_Possession_of_Weapons_24M_36M
Victim_Vehicle_Offences_24M_36M
Victim_Sexual_Offences_36M_60M
Victim_Violence_Against_The_Person_36M_60M
Victim_Possession_of_Weapons_36M_60M
Victim_Theft_36M_60M
Victim_Public_Order_Offences_36M_60M
Victim_Drug_Offences_36M_60M
Victim_Miscellaneous_Crimes_Against_Society_36M_60M
Victim_Arson_and_Criminal_Damage_36M_60M
Victim_Vehicle_Offences_36M_60M
Victim_Burglary_36M_60M
Victim_Robbery_36M_60M
Victim_DA_00M_03M
Victim_DA_00M_06M
Victim_DA_00M_12M
Victim_DA_00M_24M
Victim_DA_00M_36M
Victim_DA_00M_60M
Victim_DA_03M_06M
Victim_DA_06M_12M
Victim_DA_12M_24M
Victim_DA_24M_36M
Victim_DA_36M_60M
Offender_Violence_Against_The_Person_00M_03M
Offender_Burglary_00M_03M
Offender_Arson_and_Criminal_Damage_00M_03M
Offender_Public_Order_Offences_00M_03M
Offender_Possession_of_Weapons_00M_03M
Offender_Theft_00M_03M
Offender_Drug_Offences_00M_03M
Offender_Sexual_Offences_00M_03M
Offender_Miscellaneous_Crimes_Against_Society_00M_03M
Offender_Robbery_00M_03M
Offender_Vehicle_Offences_00M_03M
Offender_Violence_Against_The_Person_00M_06M
Offender_Burglary_00M_06M
Offender_Arson_and_Criminal_Damage_00M_06M
Offender_Public_Order_Offences_00M_06M
Offender_Possession_of_Weapons_00M_06M
Offender_Theft_00M_06M
Offender_Drug_Offences_00M_06M
Offender_Sexual_Offences_00M_06M
Offender_Robbery_00M_06M
Offender_Miscellaneous_Crimes_Against_Society_00M_06M
Offender_Vehicle_Offences_00M_06M
Offender_Sexual_Offences_00M_12M
Offender_Arson_and_Criminal_Damage_00M_12M
Offender_Possession_of_Weapons_00M_12M
Offender_Violence_Against_The_Person_00M_12M
Offender_Theft_00M_12M
Offender_Drug_Offences_00M_12M
Offender_Public_Order_Offences_00M_12M
Offender_Miscellaneous_Crimes_Against_Society_00M_12M
Offender_Burglary_00M_12M
Offender_Vehicle_Offences_00M_12M
Offender_Robbery_00M_12M
Offender_Violence_Against_The_Person_00M_24M
Offender_Theft_00M_24M
Offender_Burglary_00M_24M
Offender_Public_Order_Offences_00M_24M
Offender_Miscellaneous_Crimes_Against_Society_00M_24M
Offender_Drug_Offences_00M_24M
Offender_Sexual_Offences_00M_24M
Offender_Arson_and_Criminal_Damage_00M_24M
Offender_Possession_of_Weapons_00M_24M
Offender_Vehicle_Offences_00M_24M
Offender_Robbery_00M_24M
Offender_Violence_Against_The_Person_00M_36M
Offender_Theft_00M_36M
Offender_Burglary_00M_36M
Offender_Public_Order_Offences_00M_36M
Offender_Miscellaneous_Crimes_Against_Society_00M_36M
Offender_Drug_Offences_00M_36M
Offender_Sexual_Offences_00M_36M
Offender_Arson_and_Criminal_Damage_00M_36M
Offender_Possession_of_Weapons_00M_36M
Offender_Vehicle_Offences_00M_36M
Offender_Robbery_00M_36M
Offender_Violence_Against_The_Person_00M_60M
Offender_Theft_00M_60M
Offender_Burglary_00M_60M
Offender_Public_Order_Offences_00M_60M
Offender_Miscellaneous_Crimes_Against_Society_00M_60M
Offender_Drug_Offences_00M_60M
Offender_Sexual_Offences_00M_60M
Offender_Arson_and_Criminal_Damage_00M_60M
Offender_Possession_of_Weapons_00M_60M
Offender_Vehicle_Offences_00M_60M
Offender_Robbery_00M_60M
Offender_Violence_Against_The_Person_03M_06M
Offender_Public_Order_Offences_03M_06M
Offender_Arson_and_Criminal_Damage_03M_06M
Offender_Miscellaneous_Crimes_Against_Society_03M_06M
Offender_Sexual_Offences_03M_06M
Offender_Theft_03M_06M
Offender_Burglary_03M_06M
Offender_Robbery_03M_06M
Offender_Drug_Offences_03M_06M
Offender_Vehicle_Offences_03M_06M
Offender_Possession_of_Weapons_03M_06M
Offender_Sexual_Offences_06M_12M
Offender_Miscellaneous_Crimes_Against_Society_06M_12M
Offender_Arson_and_Criminal_Damage_06M_12M
Offender_Possession_of_Weapons_06M_12M
Offender_Violence_Against_The_Person_06M_12M
Offender_Public_Order_Offences_06M_12M
Offender_Drug_Offences_06M_12M
Offender_Theft_06M_12M
Offender_Vehicle_Offences_06M_12M
Offender_Burglary_06M_12M
Offender_Robbery_06M_12M
Offender_Violence_Against_The_Person_12M_24M
Offender_Theft_12M_24M
Offender_Burglary_12M_24M
Offender_Public_Order_Offences_12M_24M
Offender_Miscellaneous_Crimes_Against_Society_12M_24M
Offender_Drug_Offences_12M_24M
Offender_Sexual_Offences_12M_24M
Offender_Arson_and_Criminal_Damage_12M_24M
Offender_Possession_of_Weapons_12M_24M
Offender_Vehicle_Offences_12M_24M
Offender_Robbery_12M_24M
Offender_Violence_Against_The_Person_24M_36M
Offender_Drug_Offences_24M_36M
Offender_Sexual_Offences_24M_36M
Offender_Arson_and_Criminal_Damage_24M_36M
Offender_Public_Order_Offences_24M_36M
Offender_Miscellaneous_Crimes_Against_Society_24M_36M
Offender_Burglary_24M_36M
Offender_Vehicle_Offences_24M_36M
Offender_Robbery_24M_36M
Offender_Theft_24M_36M
Offender_Possession_of_Weapons_24M_36M
Offender_Sexual_Offences_36M_60M
Offender_Violence_Against_The_Person_36M_60M
Offender_Possession_of_Weapons_36M_60M
Offender_Theft_36M_60M
Offender_Public_Order_Offences_36M_60M
Offender_Drug_Offences_36M_60M
Offender_Miscellaneous_Crimes_Against_Society_36M_60M
Offender_Robbery_36M_60M
Offender_Arson_and_Criminal_Damage_36M_60M
Offender_Burglary_36M_60M
Offender_Vehicle_Offences_36M_60M
Offender_DA_00M_03M
Offender_DA_00M_06M
Offender_DA_00M_12M
Offender_DA_00M_24M
Offender_DA_00M_36M
Offender_DA_00M_60M
Offender_DA_03M_06M
Offender_DA_06M_12M
Offender_DA_12M_24M
Offender_DA_24M_36M
Offender_DA_36M_60M
Misper_Occ_00M_03M
Misper_Occ_00M_06M
Misper_Occ_00M_12M
Misper_Occ_00M_24M
Misper_Occ_00M_36M
Misper_Occ_00M_60M
Misper_Occ_03M_06M
Misper_Occ_06M_12M
Misper_Occ_12M_24M
Misper_Occ_24M_36M
Misper_Occ_36M_60M

The below list of variables are all imported at the beginning of the model process but not all are used. There is no output created for what is used as it is done as part of the process.

The weights of each variable and/or category of variable in the calculations performed by the risk model i.e. how much it contributes to the overall scores the model outputs 

Not weighted

5) training datasets of target cohorts used to train the above models 

training datasets of target cohorts used to train the above models 

Children worked with by BASE (Barnardo’s Against Child Exploitation) programme

Children worked with by Barnardo’s ROUTES programme

6) a description of the training data used by the above risk models which may include graphs showing the distributions of variables. (These descriptions should include all of the following if available:

dataset size, variable names, classification categories, the labels of classification for data points.) Please also indicate in the description if any labels are assigned by humans. 

A description of the training data is not held. The model is split into 80/20 training test, selected at random during development.

7) Manuals, instructions and any training material for use of the above risk models 

No manuals, instructions or training manuals are held.

8) Any available evaluations of the above risk models which may include model performance and receiver operating characteristic curves 

For the Risk of Sexual Exploitation Model and the Child at Risk of Criminal Exploitation Model, reviews of processes used were conducted by a third party. No reviews are held on police systems. The review could not be completed within the time frame, so no report was produced.

9) If any of above models have been decommissioned, please provide the most recent information held about that model AND provide the name of that model that replaces it. 

All of the above models have now been decommissioned. None have been replaced.