Data leakage is a situation where a machine learning model receives information during training that would not be available in real use. The model then appears to perform very well during development, but its performance can collapse when it is tested on genuinely new data or deployed in production.

Data leakage is one of the most dangerous problems in machine learning, because it can make a weak model look excellent. The evaluation metrics may look impressive, the model may seem accurate and the project may appear successful. In reality, the model has seen information it should never have had.

The easiest way to understand data leakage is to imagine a student taking an exam with access to the answer key. The score may be perfect, but it does not prove that the student understands the subject. In the same way, a model affected by data leakage may achieve high accuracy without learning a pattern that will work in the real world.

What data leakage means

Data leakage happens when information from outside the legitimate training situation enters the model-building process. This can come from the test set, from the future, from the target variable, from incorrect preprocessing or from features that indirectly reveal the answer.

The core rule is simple: if the model would not have this information when making a real prediction, it should not have it during training, feature engineering, preprocessing, model selection or evaluation.

For example, if a model predicts whether a customer will cancel a subscription, it must not use a column that contains the cancellation date. That information is known only after the event has happened. If the model uses it, it is not predicting churn. It is reading the answer.

Why data leakage is so dangerous

Data leakage is dangerous because it usually improves the numbers. This makes the problem harder to detect. A model with leakage may look more accurate, more stable and more promising than it really is.

The damage often appears later. When the model is deployed, it no longer has access to the leaked information. Its performance drops, business users lose trust and the team may not immediately understand why the model failed.

Data leakage can lead to:

• overly optimistic evaluation – the model looks better in testing than it will be in production,
• wrong business decisions – teams may trust predictions that are not reliable,
• failed deployment – the model performs poorly on real data,
• wasted development time – teams optimise a model that was never valid,
• false scientific conclusions – research results may not reproduce,
• loss of trust – users stop trusting machine learning outputs after deployment failure.

A simple example of data leakage

Imagine that you want to build a model predicting whether a customer will stop using a service next month.

The dataset contains useful columns such as account age, number of support tickets, last login date, payment history and product usage. But it also contains a column called account_closed_date. If the value is filled in, the customer has already cancelled.

If this column is included in training, the model will easily predict churn. It does not need to learn customer behaviour. It only needs to notice that the account already has a cancellation date. The model may look extremely accurate in testing, but it will fail in real use because, at prediction time, the cancellation date will not yet exist.

Training data vs real-use data

The most important question in data leakage is not: “Does this column improve the model?” The most important question is: “Would this information be available at the moment when the model makes a real prediction?”

If the answer is no, the feature is suspicious. It may be future information, post-event information, a target-derived variable or a hidden shortcut.

For example, a loan risk model should not use information that is only known after the loan has been repaid. A medical risk model should not use treatment outcomes if it is supposed to predict risk before treatment. A fraud model should not use labels created after manual investigation if those labels would not be available at the time of transaction scoring.

The prediction moment

The prediction moment is the exact time when the model is supposed to make a decision. It is one of the best practical tools for preventing data leakage.

If a model predicts tomorrow’s demand at 8:00 today, it can use information available before 8:00 today. It cannot use tomorrow’s sales, tomorrow’s stock level or a report generated after the prediction.

If a model predicts customer churn at the beginning of a month, it can use historical customer behaviour before that date. It cannot use support tickets, payment events or cancellation data created later in the same month.

Once the prediction moment is clear, invalid features become much easier to identify. The question is no longer abstract. You can simply ask whether a specific feature would really exist at that point in time.

Common types of data leakage

Data leakage can appear in several forms. Some are obvious, others are subtle. The most common types include target leakage, train-test contamination, temporal leakage and preprocessing leakage.

Target leakage

Target leakage happens when a feature contains information that directly or indirectly reveals the target variable.

If a model predicts whether a customer will complain, a column called complaint_resolved_date would be a leak. It only exists after the complaint has happened. If a model predicts whether a patient has a disease, a column containing the final diagnosis code may reveal the answer.

Target leakage can also be indirect. A feature may not contain the target itself, but it may be created using information that would only be known after the outcome.

Train-test contamination

Train-test contamination happens when information from the test set influences training. The test set is supposed to simulate new, unseen data. If it leaks into training, preprocessing or model selection, the evaluation becomes unreliable.

This can happen when data is split incorrectly, when duplicate records appear in both training and test sets, or when preprocessing is fitted on the full dataset before splitting.

Temporal leakage

Temporal leakage happens when future information is used to predict the past. This is common in time-based problems such as forecasting, churn prediction, risk scoring, demand prediction and financial modelling.

If a model predicts sales for March, it must not use information from April. If it predicts whether a customer will churn next month, it must not use events that happened after the prediction date.

Preprocessing leakage

Preprocessing leakage happens when transformations are learned from the full dataset before the train-test split. This includes scaling, imputation, encoding, feature selection, dimensionality reduction or other data preparation steps.

For example, if missing values are filled using the average value calculated from the full dataset, the training process has indirectly used information from the test set. The correct approach is to fit preprocessing only on the training data and then apply the learned transformation to validation or test data.

Data leakage through feature engineering

Feature engineering can easily create leakage if features are built without respecting the prediction moment.

Imagine a model that predicts whether a user will buy something within the next seven days. A feature called total_orders_next_7_days would clearly leak the answer. But leakage can also be less obvious. A feature such as average_order_value_after_campaign may contain information from the future if it is calculated after the prediction date.

Good feature engineering must always define the time boundary. Every feature should be calculated only from information that would have been available before or at the prediction moment.

Data leakage and feature selection

Feature selection is the process of choosing the most useful input variables for a model. It can improve model quality, reduce noise and make a model easier to interpret. But if it is done incorrectly, it can also create data leakage.

The common mistake is selecting features using the full dataset before the train-test split. If the method uses all labels to decide which features are strongest, the test set has already influenced the modelling process. The final evaluation is no longer clean.

Feature selection should be part of the training workflow. In cross-validation, feature selection should be repeated inside each fold, using only the training part of that fold. This may be slower, but it gives a more honest estimate of model performance.

Data leakage and dimensionality reduction

Dimensionality reduction methods can also leak information if they are fitted before the train-test split. PCA, for example, may not use the target variable directly, but it still learns the structure of the full dataset. If the test data influences that structure, the evaluation becomes less reliable.

The correct approach is to fit PCA or any other learned transformation on the training data only. The learned transformation can then be applied to validation or test data.

This matters especially when data changes over time. Fitting transformations on future data can make the training environment unrealistically close to the test environment.

Data leakage in time series and forecasting

Time series problems are especially sensitive to leakage. Forecasting is about predicting the future from the past. Any use of future values can make the model look unrealistically strong.

Leakage can happen when future data is used for scaling, when rolling averages are calculated incorrectly, when the train-test split is random instead of time-based or when lag features accidentally include future observations.

For example, if a demand forecasting model uses a moving average that includes future sales, it is no longer forecasting. It is using information from the period it is supposed to predict.

In time-based problems, the split should usually respect chronological order. The model should train on older data and be evaluated on later data. This better represents real deployment.

Data leakage in cross-validation

Cross-validation is used to estimate how well a model generalises. But cross-validation can also create leakage if it is applied incorrectly.

One common mistake is applying preprocessing to the full dataset before cross-validation. For example, standardising all rows before splitting them into folds means that each fold has influenced the transformation. This leaks information across folds.

The correct approach is to include preprocessing inside a pipeline. For each fold, the preprocessing step must be fitted only on the training part of that fold and then applied to the validation part.

Another risk appears with grouped data. If records from the same user, patient, device or company appear in both training and validation folds, the model may learn identity-specific patterns instead of general rules. In those cases, grouped cross-validation may be needed.

Data leakage through duplicates

Duplicate or near-duplicate records can create leakage when the same or almost the same case appears in both training and test data.

For example, if a dataset contains multiple rows for the same customer, patient, image, product or document, a random split may place related records into both sets. The model then appears to perform well because it has already seen a very similar example during training.

This is common in document classification, medical datasets, image datasets, product catalogues and customer behaviour data. The solution is to split data by entity when necessary. If the model will be used on new customers, the test set should contain customers not seen during training.

Data leakage in medical and scientific models

Data leakage is especially serious in medical and scientific machine learning because it can lead to false conclusions. A model may appear to predict disease, risk or treatment outcome, but the result may depend on hidden leakage in the dataset or workflow.

For example, a medical image model may learn scanner markings, hospital-specific artefacts or post-diagnosis codes instead of disease patterns. A clinical prediction model may accidentally use information recorded after diagnosis. A scientific model may use data splits that allow information from the same subject or experiment to appear in both training and test sets.

This is why leakage is not only an engineering issue. It can affect reproducibility, scientific validity and real-world safety.

Data leakage in marketing and e-commerce

In marketing and e-commerce, data leakage often appears in customer prediction tasks.

A churn model may use information created after the customer already left. A lead scoring model may use fields filled only after the sales team has contacted the lead. A purchase prediction model may use post-purchase behaviour. A campaign model may use metrics from the campaign period when the prediction is supposed to happen before the campaign starts.

These models can look very strong in analysis. But in real use, those fields are missing, delayed or unavailable. The model then fails because it was built on information from the wrong point in time.

Data leakage in fraud detection

Fraud detection models are also vulnerable to leakage. A model may accidentally use fields that are only created after manual review, investigation or chargeback processing.

For example, if a feature indicates that a transaction was sent to a fraud investigation team, it may reveal information that would not be available before the model’s decision. If a model is supposed to flag suspicious transactions in real time, it cannot use signals created days later.

The prediction moment matters. A real-time fraud model can use transaction amount, device information, location, merchant, customer history and recent behaviour. It cannot use the final fraud label before that label exists.

Data leakage in LLM systems

Data leakage can also appear in systems built around large language models. The form is different from classic tabular machine learning, but the principle is similar: the system gets information during evaluation or development that would not be available in a real task.

For example, an evaluation prompt may accidentally contain the correct answer. A benchmark may be too similar to examples the model has already seen. A test set may be repeatedly used while prompts, retrieval settings or system instructions are tuned. The system then becomes optimised for the test rather than for real-world use.

This is especially important for AI assistants, document analysis tools and internal knowledge systems. A model can appear strong if the evaluation setup gives it hidden clues. That does not prove that the system will handle fresh, messy and incomplete real requests.

Data leakage in RAG systems

In RAG systems, the model retrieves external documents and uses them to generate an answer. Data leakage can occur if evaluation questions are built from the same passages that are later used as retrieval targets without proper separation, or if test answers are present in the retrieval corpus in a way that would not be allowed in real use.

For example, if the evaluation asks questions whose exact answers are included in a special test document inside the retrieval index, the system may look better than it really is. It is not necessarily retrieving and reasoning well. It may only be finding a prepared answer.

A good RAG evaluation should separate source preparation, retrieval corpus, development examples and final test questions. It should also check whether the system retrieves the right evidence, not just whether the final answer sounds plausible.

Data leakage and chunking

Chunking is the process of splitting long documents into smaller parts so they can be indexed, retrieved or inserted into a model context. In RAG systems, chunking is useful, but it can also create evaluation problems if not handled carefully.

If chunks from the same original document appear in both development and evaluation sets, the system may appear to generalise while actually seeing very similar material. If the answer is split into a neighbouring chunk that is always retrieved during testing, the evaluation may be easier than the real task.

This does not mean chunking is wrong. It means document-level separation should be considered when building realistic evaluations. If the production task requires answering questions about new documents, the test should include documents not used during development.

Data leakage and prompt engineering

Prompt engineering can help make evaluation clearer, but it can also accidentally create leakage if the prompt contains the answer or gives away labels.

For example, if a prompt asks a model to classify a customer complaint but includes an internal field called escalated_to_retention_team, the model may infer that the customer is at high churn risk. If the real system would not receive that field, the evaluation is contaminated.

Good prompts for evaluation should avoid hidden answer cues. They should contain only information that would be available in the real task.

Data leakage and embeddings

Embeddings are numerical representations of content such as text, documents, products or images. Leakage can occur when embeddings are created, fine-tuned or evaluated in a way that mixes training and test information.

For example, if a document retrieval system is evaluated on test questions but the test answers are included in the indexed corpus in a form that would not exist in production, the result may be misleading. If embeddings are fine-tuned using labels from the evaluation set, the test no longer measures generalisation.

As with other machine learning workflows, the key is separation. Training, tuning, indexing, retrieval and evaluation data must be organised so that the system cannot access information it should not have.

Data leakage and overfitting

Data leakage and overfitting are related, but they are not the same thing.

• Overfitting happens when a model learns the training data too closely, including noise, random details or accidental patterns. It performs well on training data but poorly on new data.
• Data leakage happens when the training or evaluation process includes information that should not be available. The model may perform well not because it generalised, but because it was given hidden access to the answer.

Both problems create false confidence. But leakage is often more dangerous because it can make validation or test performance look excellent as well.

Data leakage vs concept drift

Data leakage should also be separated from concept drift.

Concept drift means that the relationship between inputs and outputs changes over time. A model may have been valid when trained, but becomes less accurate because the world changes.

Data leakage means that the model was evaluated incorrectly from the start because it used information it should not have had.

In production, both problems can appear together. A model may have been built with leakage and later also suffer from drift. That is why both careful validation and ongoing monitoring matter.

Data leakage and model explainability

Model explainability can help detect data leakage. If an explanation shows that a model relies heavily on a suspicious feature, this can reveal that the feature leaks the answer.

For example, a churn model may claim that the most important feature is cancellation_date. A fraud model may rely on a field created after investigation. A medical model may depend on a post-diagnosis code. These explanations are warning signs.

However, explainability does not automatically prevent leakage. A model can still leak information in ways that are hard to notice. Explanations should be used together with careful data review, time-based validation and pipeline controls.

Data leakage and bagging

Bagging can reduce variance by combining multiple models trained on different samples of the data. It is useful in many machine learning workflows, especially with unstable models such as decision trees.

But bagging does not fix data leakage. If the original dataset contains leaked features, every model in the ensemble can learn from the same invalid signal. The ensemble may even make the leaked pattern look more stable.

This is an important practical point: model architecture cannot repair a contaminated dataset. Leakage has to be fixed in the data and workflow, not hidden under a stronger algorithm.

Warning signs of data leakage

Data leakage can be difficult to find, but some warning signs should trigger review.

• Performance is unrealistically high – the model scores far better than expected for the problem.
• Validation performance is much higher than production performance – the model collapses after deployment.
• One feature dominates the model – a single variable explains almost everything.
• Feature names refer to future events – examples include cancellation, approval, resolution, payment completion or manual review.
• Random split looks good, time split looks bad – future information may be leaking into training.
• Duplicates appear across train and test sets – the model may have seen nearly identical examples.
• Preprocessing was done before splitting – scaling, imputation or selection may have used test data.

How to prevent data leakage

Preventing data leakage requires discipline in how data is prepared, split, transformed, tested and deployed. The goal is to make the modelling workflow match the real prediction situation as closely as possible.

Useful prevention practices include:

• Define the prediction moment – decide exactly when the model will make its prediction.
• Audit every feature – check whether each value would be available at that moment.
• Split data before preprocessing – fit transformations only on training data.
• Use pipelines – keep preprocessing and modelling inside cross-validation properly.
• Respect time order – use time-based splits for forecasting and temporal problems.
• Split by entity when needed – keep the same customer, patient, device or document group out of both train and test sets.
• Remove target-derived fields – exclude columns created after the outcome is known.
• Document feature availability – track when and how each feature is created.
• Compare offline and production performance – monitor whether the model behaves as expected after deployment.

Correct preprocessing workflow

A common leakage mistake is preprocessing the whole dataset before splitting. This can happen with scaling, imputation, encoding, feature selection, PCA or other transformations.

The correct workflow is:

1. Split the data into training, validation and test sets.
2. Fit preprocessing only on the training set.
3. Apply the learned transformation to validation and test sets.
4. Train the model on transformed training data.
5. Evaluate the model on transformed validation or test data.

This ensures that the test data remains unseen. The model and preprocessing steps do not get to learn from it.

How to investigate suspected leakage

If you suspect data leakage, do not start by tuning the model. Start by checking the data and workflow.

• Review feature names – look for variables that refer to future outcomes or post-event processes.
• Check feature creation dates – verify when each feature becomes available.
• Inspect feature importance – suspiciously dominant features may reveal leakage.
• Use time-based validation – compare with random splits to identify temporal leakage.
• Search for duplicates – check whether related records appear in both train and test data.
• Move preprocessing into a pipeline – prevent transformations from learning from evaluation data.
• Rebuild the model without suspicious features – see whether performance drops to a more realistic level.

A large performance drop after removing suspicious fields is not always bad news. It may mean that the model is finally being evaluated honestly.

Data leakage in production systems

Data leakage can also appear after deployment if training and production environments are inconsistent.

For example, a model may be trained using a feature that is technically available in historical data but delayed in production. During training, the feature looks valid. In real time, it is missing or incomplete. The model then receives different information from what it saw during development.

This is why production feature availability matters. A feature is not valid only because it exists somewhere in the database. It must exist at the right time, in the right format and with the same meaning in production.

Data leakage and model governance

Data leakage is easier to prevent when feature definitions, data sources and model workflows are documented. In larger organisations, this becomes part of model governance.

Teams should know where each feature comes from, when it is created, how it is updated, whether it depends on future events and whether it will be available in production.

For high-impact models, this documentation is not bureaucracy. It is part of risk control. Without it, a model can look valid in a notebook and fail in the real system.

Common misunderstandings about data leakage

Data leakage is often misunderstood because it does not always look like an obvious mistake. Some teams think leakage only means accidentally putting the target column into the features. In reality, leakage can be much subtler.

• Leakage is not only the target column – derived, delayed or post-event features can also leak information.
• Good test metrics do not rule out leakage – leakage can improve test scores.
• Preprocessing can leak data – scaling, imputation and feature selection can contaminate evaluation.
• Random splitting is not always valid – time-based or grouped data often needs special splitting.
• A feature can exist in the database and still be invalid – availability at prediction time is what matters.
• Leakage can happen without bad intention – it is often an accidental workflow error.

How to remember data leakage

Data leakage can be remembered as giving the model a hidden answer key. The model may look smart, but it is not solving the same problem it will face in real use.

Before using any feature, ask whether a real deployed model would know it at the moment of prediction. Before trusting any evaluation, ask whether the test data was truly kept separate from training, preprocessing and model selection.

Related terms

• Machine learning – the broader field in which models learn patterns from data and use them for predictions, classifications or decisions.
• Feature selection – the process of selecting useful input variables. It can cause leakage if the selection is done before data is split correctly.
• Large language model (LLM) – a language-focused AI model. Data leakage can appear in LLM evaluation when answers, labels or evaluation content are accidentally present in the prompt, retrieval corpus or training data.
• RAG – retrieval-augmented generation. In RAG systems, leakage can occur when evaluation answers are included in the retrieval corpus in a way that would not be valid in real use.
• Chunking – splitting longer content into smaller parts. In evaluation, chunking must be handled carefully so the system is not tested on near-duplicate content it has already seen.
• Prompt engineering – the practice of designing prompts for language models. Poorly designed evaluation prompts can accidentally include answer cues and create leakage.
• Embedding – a numerical representation of content such as text, images or documents. Leakage can occur if embeddings, indexes or evaluation sets are built with improper separation.
• Bagging – an ensemble method that can reduce variance, but it cannot repair leaked or contaminated input data.
• Target leakage – leakage caused by features that directly or indirectly reveal the target variable.
• Train-test contamination – leakage caused by test data influencing training, preprocessing or model selection.
• Temporal leakage – leakage caused by using future information to predict an earlier point in time.
• Feature engineering – creating or transforming input variables. It can introduce leakage if features use future or post-event information.
• Dimensionality reduction – reducing the number of variables or dimensions. Learned transformations such as PCA must be fitted only on training data in modelling workflows.
• Overfitting – a model learning training data too closely. It differs from leakage, but both can lead to poor real-world performance.
• Concept drift – a change in the relationship between inputs and outputs over time after deployment.
• Model monitoring – checking whether a deployed model continues to behave as expected in real use.

Sources and further reading

• What is Data Leakage in Machine Learning? – ibm.com – June 2026 – explains data leakage as information used during training that would not be available at prediction time, and describes why it causes unreliable real-world results.
• Common pitfalls and recommended practices – scikit-learn.org – June 2026 – describes common machine learning workflow mistakes, including preprocessing leakage and the need to split data before fitting transformations.
• Leakage and the Reproducibility Crisis in ML-based Science – arxiv.org – June 2026 – analyses data leakage as a major reproducibility problem in machine learning based scientific research.
• Leakage and the Reproducibility Crisis in ML-based Science – reproducible.cs.princeton.edu – June 2026 – project page from Princeton researchers summarising how leakage can distort scientific machine learning results.
• Data leakage detection in machine learning code – ncbi.nlm.nih.gov – June 2026 – discusses code-level leakage patterns and approaches for detecting leakage in machine learning workflows.
• Data Leakage – kaggle.com – June 2026 – a practical tutorial showing common leakage examples and why models may look accurate when they use information unavailable in real deployment.
• 3 subtle ways data leakage can ruin your models – machinelearningmastery.com – June 2026 – practical examples of target leakage, train-test contamination and temporal leakage in applied machine learning.