Zubair Khalid

Virologist/Molecular Biologist | Veterinarian | Bioinformatician

Conventional & Molecular Virology • Vaccine Development • Computational Biology

Dr. Zubair Khalid is a veterinarian and virologist specializing in conventional and molecular virology, vaccine development, and computational biology. Dedicated to advancing animal health through innovative research and multi-omics approaches.

Dr. Zubair Khalid - Veterinarian, Virologist, and Vaccine Development Researcher specializing in Computational Biology, Multi-omics, Animal Health, and Infectious Disease Research

Section: Infrastructure, Cloud & Policy

Radiomics Feature Selection: Methods and Best Practices

Radiomics feature selection is the process of identifying the most informative and reproducible quantitative imaging features from a high-dimensional feature set to build predictive models that generalize beyond the training data. For students, researchers, analysts, and life-science professionals working with medical images, feature selection is not an optional preprocessing step but a core methodological decision that determines whether a radiomics model will perform reliably on new patients, new scanners, or new institutions. This article explains why feature selection matters in radiomics, compares the main selection methods including LASSO, mRMR, Boruta, and graph-based approaches, and provides a practical workflow for selecting and validating features in your own studies.

What Are Radiomics Features

Radiomics features are quantitative measurements extracted from medical images that describe tumor intensity, shape, texture, and spatial relationships. These features convert visual information from CT, MRI, PET, and ultrasound into numerical values that can be analyzed with machine learning algorithms. The EMBL-EBI Training resources describe how quantitative image analysis fits within the broader bioinformatics landscape, where extracting meaningful measurements from raw data is a foundational step.

Common radiomics feature classes include first-order statistics that describe the distribution of pixel intensities, shape features that capture geometric properties of the region of interest, and texture features that quantify spatial patterns. Texture features are further divided into families such as Gray Level Co-occurrence Matrix (GLCM), Gray Level Run Length Matrix (GLRLM), Gray Level Size Zone Matrix (GLSZM), and Neighborhood Gray Tone Difference Matrix (NGTDM). Each family captures a different aspect of image texture, and the choice of which families to include affects both the dimensionality and the interpretability of the feature set.

The number of features extracted from a single image can range from dozens to thousands, depending on the software, the number of image filters applied, and whether wavelet or other transformations are included. A typical radiomics study using PyRadiomics might extract over 100 features per image, and when multiple image types or time points are analyzed, the total can exceed 3,000 features. This high dimensionality creates a situation where the number of features approaches or exceeds the number of patients, which is a fundamental challenge for predictive modeling.

Why Feature Selection Is Essential in Radiomics

Feature selection addresses several problems that arise when building predictive models from high-dimensional radiomics data. The first problem is redundancy. Many radiomics features are highly correlated with each other because they measure related aspects of the same underlying image properties. Including correlated features in a model can lead to unstable coefficient estimates and poor generalization.

The second problem is irrelevance. Many extracted features carry no information about the clinical outcome of interest. A study of radiomics-based prediction modeling noted that since many quantitative features are non-informative, feature selection becomes essential for building useful models. The flexible feature selection framework study demonstrated that controlling relevance and redundancy among features allows researchers to explore optimal combinations and evaluate predictive performance using only selected features.

The third problem is overfitting. When the number of features greatly exceeds the number of patients, a model can memorize the training data instead of learn generalizable patterns. This leads to excellent performance on the training set but poor performance on new data. Feature selection reduces the dimensionality and forces the model to focus on the most informative features, which improves the chance of generalization.

The fourth problem is reproducibility. Radiomics features are sensitive to variations in image acquisition, reconstruction, and segmentation. A study on MRI preprocessing methods found that preprocessing pipelines significantly impact feature reproducibility and that excluding non-reproducible features generally improved classification performance. Selecting features that are stable across imaging variations is critical for clinical translation.

At a Glance: Feature Selection Methods Compared

Method Category Strengths Limitations Typical Use Case
LASSO Embedded Shrinks coefficients to zero, handles correlated features, built into regression models May select unstable features across datasets, requires tuning of penalty parameter Survival analysis and binary classification with moderate feature sets
mRMR Filter Balances relevance and redundancy, computationally efficient, model agnostic Does not consider feature interactions with the model, may miss complementary features Initial screening of large feature sets before modeling
Boruta Wrapper Identifies all features relevant to the outcome, uses random forest importance Computationally intensive, may retain more features than needed Exploratory analysis when the goal is understanding instead of parsimony
Graph-FS Filter Models feature interdependencies, improves stability across institutions Newer method with less established literature, requires graph construction Multi-center studies where stability across sites is critical
Recursive Feature Elimination Wrapper Iteratively removes least important features, works with any model Computationally expensive, can overfit with small samples Studies with moderate feature counts and sufficient sample size
Extremely Randomized Trees Embedded Handles non-linear relationships, provides feature importance scores May be unstable with small samples, requires parameter tuning Classification tasks with complex feature interactions

The benchmarking study of feature projection methods compared nine selection methods including MRMRe, Extremely Randomized Trees, and LASSO against nine projection methods across 50 binary classification radiomic datasets. Selection methods, particularly ET, MRMRe, Boruta, and LASSO, achieved the highest overall performance. However, the average difference between selection and projection methods was negligible and statistically insignificant, suggesting that both approaches perform similarly when averaged across diverse datasets.

Core Principles of Radiomics Feature Selection

Stability Before Predictive Performance

The most important principle in radiomics feature selection is that stability should be evaluated before predictive performance. A feature that is highly predictive in one dataset but disappears when the imaging protocol changes is not a reliable biomarker. The graph-based feature selection study demonstrated this point by analyzing 1,648 radiomic features from 752 head and neck cancer patients across three institutions. Traditional methods including Boruta, LASSO, RFE, and mRMR achieved Jaccard Index values between 0.005 and 0.014, indicating almost no overlap in selected features across institutions. The Graph-FS approach achieved a Jaccard Index of 0.46, showing substantially better stability.

Reproducibility Filtering as a Prerequisite

Before any outcome-based feature selection, features should be filtered based on their reproducibility. Reproducibility can be assessed through test-retest scans, segmentation variability, or across reconstruction settings. The study on radiomics feature stability in photon-counting CT found that feature stability varies between organs and depends on region of interest size. Features such as glcm_JointEntropy, gldm_GrayLevelNonUniformity, and firstorder_Entropy were identified as stable across monoenergetic reconstructions.

The study on image enhancement in nasopharyngeal carcinoma MRI showed that segmentation methods significantly affect feature reproducibility. Manual segmentation achieved ICC values of 0.787 for T2-weighted images, while semi-automatic segmentation with histogram matching achieved 0.990. This finding supports the use of standardized segmentation protocols before feature selection.

The Curse of Dimensionality

Radiomics datasets typically have far more features than patients. A study on pseudoprogression prediction in glioma analyzed 3,404 radiomic features from 222 patients. After reproducibility filtering and LASSO selection, only 17 features were retained for the final model. This dramatic reduction illustrates the need for aggressive feature selection when dimensionality is high.

The study on lymphoma PET radiomics extracted 107 features per lesion from 80 patients and preselected 30 features based on robustness to variations in tracer uptake time, reconstruction parameters, and respiratory motion. This two-stage approach, where stability filtering precedes outcome-based selection, is a common and recommended pattern.

Feature Selection Methods in Detail

LASSO (Least Absolute Shrinkage and Selection Operator)

LASSO is an embedded method that performs both regularization and feature selection simultaneously. It adds a penalty term to the regression objective that shrinks some coefficients to exactly zero, effectively removing those features from the model. The amount of shrinkage is controlled by a penalty parameter that must be tuned, typically through cross-validation.

LASSO is widely used in radiomics because it handles correlated features and produces sparse models that are easy to interpret. The lymphoma PET study found that the best predictive performance was achieved using median value aggregation combined with LASSO feature selection. The glioma pseudoprogression study used LASSO in combination with univariate analysis, collinearity assessment, and random forest to select features for predicting pathological complete response in hepatocellular carcinoma.

A limitation of LASSO is that it can select different features when the data are slightly perturbed, which is problematic for multi-center studies. When the number of features is very large relative to the sample size, LASSO may also select features that are not reproducible across imaging variations.

mRMR (Minimum Redundancy Maximum Relevance)

mRMR is a filter method that selects features based on two criteria: maximum relevance to the target variable and minimum redundancy among selected features. The algorithm iteratively adds features that have high relevance to the outcome but low similarity to features already selected.

mRMR is computationally efficient and model agnostic, meaning it can be used before any classifier or regression model. The benchmarking study included MRMRe among the top-performing selection methods. However, mRMR does not consider how features interact within a specific model, so it may miss complementary features that are individually weak but jointly informative.

Boruta

Boruta is a wrapper method that uses random forest importance scores to determine whether a feature is relevant to the outcome. It creates shadow features by shuffling the original features and compares the importance of real features to the maximum importance of shadow features. Features that consistently outperform the shadow features are deemed relevant.

Boruta is useful when the goal is to identify all features that carry information about the outcome, instead of finding the smallest predictive set. The CBCT phantom study used Boruta random-forest analysis to rank features by their importance in distinguishing between phantom groups. A limitation of Boruta is that it can retain many features, which may still require further selection for building parsimonious models.

Graph-Based Feature Selection

Graph-FS is a newer approach that models feature interdependencies as a graph structure and selects features that are stable across datasets. The multi-institution head and neck cancer study demonstrated that Graph-FS achieved substantially higher stability than traditional methods, with a Jaccard Index of 0.46 compared to 0.005 to 0.014 for Boruta, LASSO, RFE, and mRMR.

The advantage of graph-based methods is that they explicitly account for relationships between features, which can improve stability when imaging protocols vary across institutions. This approach is particularly relevant for multi-center studies where the goal is to develop biomarkers that work across different scanners and protocols.

Recursive Feature Elimination

RFE is a wrapper method that recursively removes the least important features based on model coefficients or feature importance scores. The model is trained on the full feature set, the least important features are removed, and the process is repeated until the desired number of features is reached.

RFE works with any model that provides feature importance, including support vector machines and random forests. However, it is computationally expensive because it requires training multiple models. The glioma grading comparison study compared feature selection methods and machine learning classifiers for radiomics analysis, providing a reference for how RFE performs relative to other methods.

Feature Projection Methods

Feature projection methods such as Principal Component Analysis (PCA) transform the original features into a new set of uncorrelated variables. These methods are sometimes avoided in radiomics because the projected features lack direct interpretability. However, the benchmarking study noted that since most radiomic features lack inherent semantic meaning, prioritizing interpretability over predictive performance may not be justified.

The study found that projection methods, particularly Non-Negative Matrix Factorization, occasionally outperformed all selection methods on individual datasets. However, the average difference between selection and projection methods was negligible and statistically insignificant. This finding suggests that projection methods can be a reasonable alternative when interpretability of individual features is not a priority.

Practical Workflow for Feature Selection

Step 1: Define the Clinical Question and Outcome

Before any feature selection, define the clinical question and the outcome variable. The outcome could be binary such as treatment response or disease presence, time-to-event such as survival or progression, or continuous such as tumor volume change. The outcome definition determines which feature selection methods are appropriate and how performance should be evaluated.

Step 2: Extract Features with Standardized Software

Use established radiomics software that complies with the Image Biomarker Standardisation Initiative (IBSI). The study on open-source software reproducibility compared PyRadiomics, IBEX, and MaZda and found that only a subset of features were reproducible across software packages. Using IBSI-compliant software such as PyRadiomics improves the chance that your features can be compared with other studies.

Step 3: Assess Feature Reproducibility

Before outcome-based selection, assess feature reproducibility using test-retest data, segmentation variability, or phantom studies. Features with low reproducibility should be excluded regardless of their predictive potential. The study on deep learning reconstruction found that only 5% of features combined consistency, discriminative power, and repeatability with filtered back projection, while deep learning reconstruction more than doubled the yield of usable features.

The study on cardiac photon-counting CT defined stable features as those with an intraclass correlation coefficient higher than 0.75. Only 24 of 105 features showed stability in both the myocardium and epicardial adipose tissue, highlighting the importance of structure-specific stability assessment.

Step 4: Split Data Before Any Selection

A critical methodological rule is to split the data into training and test sets before any feature selection or normalization. The glioma pseudoprogression study used a strict split-first workflow where dataset partitioning preceded ICC filtering, Z-score normalization, and LASSO modeling. This prevents information leakage from the test set into the feature selection process.

Step 5: Apply Reproducibility Filtering

Filter features based on reproducibility metrics such as ICC or concordance correlation coefficient. The threshold for stability depends on the study context, but values of 0.75 or higher are commonly used. The Parkinson's disease study found that wavelet-based features showed the highest reproducibility, with 37% demonstrating excellent ICC values of 0.90 or higher.

Step 6: Apply Outcome-Based Feature Selection

Apply one or more feature selection methods to the training data only. Common approaches include:

  • Univariate screening followed by multivariate selection
  • LASSO with cross-validation to tune the penalty parameter
  • mRMR for filter-based selection
  • Boruta for identifying all relevant features
  • Graph-FS for multi-center stability

The hepatocellular carcinoma study used univariate analysis, collinearity assessment, LASSO, and random forest in combination to perform feature selection. This multi-method approach provides a robustness check on the selected features.

Step 7: Evaluate Stability of Selected Features

Assess whether the selected features are stable across different subsets of the training data. This can be done through bootstrap resampling or repeated cross-validation. The graph-based feature selection study used the Jaccard Index, Dice-Sorensen Index, and Kendall's Coefficient of Concordance to evaluate feature selection stability and ranking consistency.

Step 8: Build and Validate the Predictive Model

Train the final model using only the selected features and evaluate performance on the held-out test set. Report performance metrics appropriate for the outcome type, such as AUC for binary outcomes, C-index for survival outcomes, and calibration metrics for risk prediction. The lymphoma PET study reported Harrell's concordance indices ranging from 0.582 to 0.668 for progression-free survival and 0.597 to 0.721 for overall survival across different aggregation and selection strategies.

Step 9: Report According to Established Guidelines

Report the feature selection process in sufficient detail to allow replication. The CLEAR checklist provides a 58-item documentation standard for radiomics research that covers the minimum requirements for presenting clinical radiomics studies. Following such guidelines improves the reproducibility and credibility of your work.

Records and Measurements for Feature Selection

Feature Stability Metrics

Record the stability of each feature across the sources of variation relevant to your study. Common metrics include:

  • Intraclass correlation coefficient (ICC) for test-retest or segmentation variability
  • Concordance correlation coefficient (CCC) for agreement between measurements
  • Jaccard Index for overlap of selected feature sets across datasets
  • Dice-Sorensen Index for similarity of selected feature sets
  • Kendall's Coefficient of Concordance for ranking consistency

The study on virtual non-contrast reconstructions reported ICC values for individual features, such as glrlm_GrayLevelVariance with an ICC of 0.98 and ngtdm_Strength with an ICC of 0.97. Recording feature-specific stability values allows other researchers to assess which features are likely to be reliable in their own data.

Selection Frequency

When using resampling methods, record how often each feature is selected across bootstrap samples or cross-validation folds. Features that are selected frequently are more likely to be stable and informative. The graph-based feature selection study reported selection overlap percentages, with Graph-FS achieving 45.8% overlap compared to near-zero values for traditional methods.

Model Performance Metrics

Record performance metrics for models built with different feature sets to document the impact of feature selection choices. The benchmarking study used AUC, area under the precision-recall curve, and F1, F0.5, and F2 scores to compare selection and projection methods. The echocardiography study reported sensitivity and F1-score improvements from radiomics-based models compared to conventional parameters.

Harmonization Records

If you apply harmonization methods such as ComBat or CovBat to reduce batch effects, record the parameters used and the effect on feature variability. The CovBat harmonization study found that harmonization reduced feature variability due to hardware differences from 12.32 to 25.38% down to 1.19 to 2.01% and improved model AUC from 0.93 to 1.00 for the combined model.

Common Failure Patterns in Feature Selection

Information Leakage

The most serious failure pattern is information leakage, where the test set influences feature selection. This occurs when feature selection is performed on the entire dataset before splitting, or when normalization parameters are calculated using test data. The glioma pseudoprogression study emphasized the importance of a split-first workflow to prevent this error.

Overfitting to Training Data

Selecting too many features relative to the sample size leads to overfitting. A model with 50 features and 50 patients can achieve perfect training performance but will likely fail on new data. The flexible feature selection framework study demonstrated that the number of selected features varied across combinations, and the highest test AUCs ranged from 0.738 to 0.846 across four datasets, showing that performance depends on the balance between feature count and sample size.

Ignoring Feature Stability

Selecting features based solely on predictive performance without assessing stability leads to models that fail when applied to new imaging protocols or institutions. The multi-institution head and neck study showed that traditional selection methods achieved near-zero overlap in selected features across institutions, meaning that models built at one institution would not transfer to another.

Using Inconsistent Segmentation

Feature values depend heavily on how the region of interest is segmented. The nasopharyngeal carcinoma study found that manual segmentation achieved ICC values of 0.787 to 0.801, while semi-automatic segmentation with histogram matching achieved 0.987 to 0.990. Using inconsistent segmentation methods across patients or institutions introduces noise that feature selection cannot fully correct.

Ignoring Image Quality Variation

Image quality affects feature values and therefore feature selection. The review on image quality impact noted that variation in input image quality is one of the main sources impacting the reproducibility of radiomics analysis. Scanner-related factors such as acquisition and reconstruction settings, and patient-related factors such as motion, both affect feature values.

Comparing Methods Without Proper Validation

Comparing feature selection methods using the same test data that was used for selection leads to biased comparisons. Use nested cross-validation or a completely held-out test set for method comparison. The benchmarking study used nested, stratified 5-fold cross-validation with 10 repeats to compare selection and projection methods, providing a rigorous evaluation framework.

Limitations and Interpretation Boundaries

Feature Selection Does Not Fix Poor Image Quality

Feature selection can identify the most informative features from a given dataset, but it cannot compensate for images that are acquired with inconsistent protocols or poor quality. The CBCT phantom study noted that clinical applications of radiomics using CBCT imaging have been hindered by lack of stability in radiomics features and comparably poor image quality. Standardizing image acquisition and reconstruction is a prerequisite for meaningful feature selection.

Selected Features Are Dataset-Specific

Features selected in one study may not be relevant in another study with different imaging protocols, patient populations, or outcomes. The lymphoma PET study found that the prognostic value of radiomics remained consistent across different feature aggregation and selection strategies, but the specific features selected varied. This means that radiomics signatures should be validated externally before clinical use.

Performance Varies Across Datasets

The benchmarking study found that performance varied considerably across datasets, and some projection methods occasionally outperformed all selection methods on individual datasets. This variability means that no single feature selection method is universally best, and method choice should be guided by the specific characteristics of your data.

Interpretability Tradeoffs

Feature selection methods that preserve individual features, such as LASSO and mRMR, maintain interpretability because the selected features can be traced back to specific image properties. Projection methods such as PCA create new features that are combinations of original features, which complicates interpretation. However, the benchmarking study noted that since most radiomic features lack inherent semantic meaning, the interpretability advantage of selection methods may be limited.

Harmonization Is Not a Substitute for Standardization

Harmonization methods such as ComBat and CovBat can reduce batch effects, but they do not eliminate the need for standardized image acquisition. The CovBat harmonization study showed that harmonization improved feature consistency and model performance, but the study also noted that harmonization requires careful validation and should not be used as a substitute for protocol standardization.

Safety and Regulatory Context

Data Sharing and Privacy

Radiomics studies involve patient imaging data that must be handled according to applicable privacy regulations. The NIH Genomic Data Sharing Policy provides a framework for responsible data sharing that is relevant to radiomics research, particularly when imaging data are linked to genomic or clinical data. Researchers should ensure that their data handling procedures comply with institutional and regulatory requirements.

Data Resources and Reproducibility

The NCBI Data Resources provide access to databases and tools that support reproducible research. For radiomics studies, documenting the data sources, preprocessing steps, and feature selection methods is essential for reproducibility. The FAIR Guiding Principles describe best practices for making data findable, accessible, interoperable, and reusable, which are directly applicable to radiomics feature sets and models.

Clinical Translation Requirements

The review on radiomics and artificial intelligence noted that the current use of radiomics is mostly limited to academic research without applications in routine clinical practice. The CLEAR checklist was developed to improve the quality and reliability of radiomics research and to facilitate the repeatability and reproducibility of studies. Researchers developing radiomics models for clinical use should follow established reporting guidelines and validate their models on independent external datasets.

Professional Escalation Criteria

When to Seek Statistical Expertise

If your dataset has more than 100 features per 10 patients, or if you are unsure whether your feature selection method is appropriate for your data structure, consult a biostatistician or data scientist with experience in high-dimensional data analysis. The flexible feature selection framework study noted that feature selection methods have been mixed with filter, wrapper, and embedded methods without a rule of thumb, indicating that expert guidance is often needed.

When to Question Your Results

If your model achieves an AUC above 0.95 on the test set, be suspicious of information leakage or overfitting. If selected features change dramatically when you add or remove a few patients, your feature selection is unstable. If your model performs well on internal validation but poorly on external validation, your features may not be reproducible across imaging protocols.

When to Reconsider the Study Design

If your imaging data come from multiple institutions without harmonization, or if your segmentation methods vary across patients, reconsider the study design before proceeding with feature selection. The multi-institution head and neck study demonstrated that traditional feature selection methods achieve near-zero stability across institutions, which means that models built without addressing multi-center variability will not generalize.

Frequently Asked Questions

What is the difference between feature selection and feature extraction in radiomics?

Feature selection chooses a subset of the original extracted features, preserving their identity and interpretability. Feature extraction, also called feature projection, transforms the original features into new variables, such as principal components, that are combinations of the original features. The benchmarking study compared these two approaches and found that selection methods achieved the highest overall performance, though projection methods occasionally outperformed selection on individual datasets.

How many radiomics features should I select for my model?

There is no universal number, but a common rule of thumb is to have at least 10 to 20 events or outcomes per selected feature to avoid overfitting. The glioma pseudoprogression study retained 17 features from 3,404 initial features for a model with 222 patients. The lymphoma PET study preselected 30 features from 107 for a study with 80 patients. The appropriate number depends on your sample size, outcome type, and model complexity.

What is the best feature selection method for radiomics?

No single method is universally best. The benchmarking study found that ET, MRMRe, Boruta, and LASSO achieved the highest overall performance, but performance varied considerably across datasets. The graph-based feature selection study showed that Graph-FS achieved better stability across institutions than traditional methods. Choose methods based on your sample size, outcome type, and whether stability across centers is a priority.

How do I assess the stability of selected radiomics features?

Assess stability using test-retest data, segmentation variability, or bootstrap resampling. Common metrics include the intraclass correlation coefficient, concordance correlation coefficient, Jaccard Index, and Dice-Sorensen Index. The multi-institution head and neck study used these metrics to compare feature selection stability across institutions and found that Graph-FS achieved a Jaccard Index of 0.46 compared to 0.005 to 0.014 for traditional methods.

Should I use LASSO or mRMR for radiomics feature selection?

Both methods are effective but serve different purposes. LASSO is an embedded method that performs selection within a regression framework and handles correlated features, making it suitable for building predictive models. mRMR is a filter method that balances relevance and redundancy independently of any model, making it suitable for initial screening of large feature sets. The lymphoma PET study found that LASSO achieved the best predictive performance, while the benchmarking study found that both MRMRe and LASSO were among the top-performing methods.

What is the role of harmonization in feature selection?

Harmonization methods such as ComBat and CovBat reduce feature variability caused by different imaging devices or protocols. The CovBat harmonization study found that harmonization increased the number of consistent features by 68.82% with ComBat and 73.12% with CovBat, and improved model AUC from 0.93 to 1.00. Harmonization should be applied before feature selection to ensure that selected features are stable across centers.

Can I use deep learning features instead of handcrafted radiomics features?

Deep learning can extract features automatically from images, but these features are different from handcrafted radiomics features. The breast MRI study compared handcrafted radiomics with frozen deep image embeddings and found that the strongest radiomics model achieved an AUROC of 0.670, while the best deep embedding model used delta features with 30 selected features. The choice between handcrafted and deep features depends on your data size, computational resources, and interpretability requirements.

How do I report feature selection methods in my radiomics paper?

Report the feature selection methods, parameters, and validation approach in sufficient detail to allow replication. The CLEAR checklist provides a 58-item documentation standard for radiomics research that covers feature selection and other methodological steps. Include the number of features before and after selection, the stability metrics used, and the performance of models built with different feature sets.

Related Bioinformatics Guides

References and Further Reading

This article is educational and does not replace validated analysis plans, institutional policy, clinical interpretation, or specialist review.