EOSC-SIESTA (Secure Interactive Environments for SensiTive data Analytics) is a European project, funded under the Horizon Europe programme, that strengthens the European Open Science Cloud (EOSC) with trusted, cloud-based environments for managing and analysing sensitive data. Its goal is to give researchers practical, reproducible ways of working with confidential datasets while still respecting FAIR principles, so that data can remain findable, accessible, interoperable, and reusable even when it cannot be fully opened to the public.
All of these tools are brought together in a single entry point: the SIESTA Dashboard, the project’s main interface, where users can discover and launch the different services on offer.
One of these tools available through the Dashboard is DatLeak, a tool for the detection of data leakage in a differentially private dataset. DatLeak is being developed by the Neurobiology Research Unit (NRU) at Copenhagen University Hospital.
Randomizing or scrambling a dataset does not automatically make it safe. If the transformation is not applied carefully, the resulting variables can still carry traces of the original values, that could be used to link a record back to a real participant. DatLeak was built precisely to detect that risk, by checking whether a scrambled dataset still leaks information from the original.
DatLeak addresses this by directly comparing two versions of the same dataset, the original and its scrambled or synthetic counterpart, and quantifying exactly how much information has leaked from one into the other. It currently supports two very different data types: tabular datasets and neuroimaging data.
Types of data leakage
DatLeak works with the same underlying idea regardless of the type of data involved: a row, an image, or a signal can leak information either completely or partially. What changes between data types is how that similarity is actually measured, since comparing two spreadsheet cells is a very different task from comparing two brain scans.
For tabular data, the comparison is direct: it checks each row of the scrambled dataset against the same row in the original, cell by cell. A row is marked as full leakage when every valid cell in that row still matches the original, meaning the anonymization had no effect at all for that participant. A row is marked as partial leakage when only some, but not all, of its cells match, meaning the row was altered but still retained some original values. The tool reports both as percentages across the dataset, along with the average number of matching cells per row.
For neuroimaging, DatLeak combines three techniques to assess how closely a scrambled image resembles the original: an exact-match check, the Pearson correlation coefficient, and the Structural Similarity Index Measure (SSIM), applied across the image’s spatial dimensions and, where relevant, over time. Full leakage is recorded when a scrambled image is essentially identical to the original, an almost perfect correlation confirmed by an exact match. Partial leakage is calculated instead from the strongest correlations found between the original and scrambled data, capturing cases where no exact copy exists but a meaningful trace of the original structure has survived.
How DatLeak Works
DatLeak’s code is publicly available in its repository, allowing researchers to inspect the analysis step by step, see exactly how the comparisons are computed, and adapt the parameters to their own data.
For tabular datasets, running DatLeak is as simple as pointing it at the original and scrambled CSV or TSV files from the command line, optionally telling it which values or columns to ignore in the comparison. For neuroimaging datasets, the tool is pointed at the original and scrambled image directories instead, and can optionally generate a detailed HTML report with side-by-side visual snapshots, correlation and similarity statistics, and distribution plots that make it easy to spot where leakage is occurring.
The neuroimaging analysis of DatLeak supports four types of brain data: 3D structural T1-weighted fMRI, 4D fMRI image, MEG (Magnetoencephalography) and EEG (Electroencephalography).
DatLeak is also accessible through the SIESTA Dashboard, which provides direct access to a Jupyter environment, allowing researchers to explore the tool interactively without installing anything locally.
Tests and examples
One of the neuroimaging examples available in the repository is use case 2.2, which uses a 3D structural T1-weighted fMRI image downloaded from OpenNeuro. It evaluates six different scrambling methods implemented in BIDScramble: Original vs. Original as a baseline, Permutation, Permutation with slices replaced, Wobble, Scatter, and Blur, each tested at increasing levels of intensity.
For each test, the results are visualized as plots with the scrambling parameter on the x-axis and the leakage score on the y-axis, allowing researchers to track how leakage evolves as the scrambling intensity increases.
For tabular data, the documentation includes a worked example using sample original and scrambled files, which illustrates the command-line syntax and the kind of output DatLeak produces, including the percentage of full and partial leakage and the average number of matching cells per row.