Quantitative Imaging Analysis
In medical applications, multiple images are acquired from the same subject at different times or from different subjects. The critical stage for utilizing these images is to align them in order to visualize the combined information. Image registration includes the processing to resolve the mapping between images so that the features or structures in one image correspond to those in the other image. The transformation between two image scenes can be of either rigid or non-rigid. Rigid body transformation has six degrees of freedom in three dimensions, i.e. three translations and three rotations. For most body organs, the motion is non-rigid and requires more degrees of freedom to accurately describe tissue motion. Therefore, deformable registration becomes necessary for many image applications. One example is the registration of pre- and post-contrast enhanced breast MRI images. Deformable registration is required for this application as soft tissue, such as breast tissue, always undergoes non-rigid motion between images. Other similar applications can be found during medical imaging diagnosis where different modality images or the same modality at different times, are acquired and require deformable registration because of the non-rigid tissue deformation between images. Examples include heart imaging and chest PET-CT imaging where non-rigid motion can be a major tissue motion source. In image-guided radiation therapy, because of the type of treatment or patient respiration, a non-rigid shape or position change in an organ is unavoidable. Deformable image registration is critical for delivering an appropriate radiation dose and in order to avoid the damage to adjacent healthy tissue. An example is image-guided radiation therapy of prostate tumors or tumors in other organs.
Thin-Plate Spline (TPS) Deformable Image Registration
We created and evaluated an almost fully automated, 3D non-rigid registration algorithm using mutual information and a thin-plate spline (TPS) transformation for MR images of the prostate and pelvis. In the first step, an automatic rigid-body registration with special features was used to capture the global transformation. In the second step, local feature points were registered. An operator entered only five feature points (FPs) located at the prostate center, in the left and right hip joints, and in the left and right distal femurs. The program automatically determined and optimized other FPs on the external pelvic skin surface and along the femurs. More than 600 control points were used to establish a TPS transformation for deformation of the pelvic region and the prostate. Ten volume pairs were acquired from three volunteers in the diagnostic (supine) and treatment positions (supine with legs raised). Various visualization techniques showed that warping rectified the significant pelvic misalignment caused by the rigid body method. Gray-value measures of registration quality, including mutual information, correlation coefficient, and intensity difference, all improved with warping. The distance between prostate 3D centroids was 0.7 ± 0.2 mm following warping compared to 4.9 ± 3.4 mm with rigid-body registration. The semiautomatic non-rigid registration works better than rigid body registration when the patient position is changes significantly between acquisitions; it could be a useful tool for many applications of prostate diagnosis and therapy.
B-Spline Deformable Image Registration
We implemented mutual information B-spline deformation registration algorithms. Mutual information does not assume a linear intensity relationship between images and has been used for registration of images of either the same modality or different modalities. A motion constraint is optimized in order to achieve a smooth function instead of an unrealistic result. A gradient-based minimization method is used to find the B-splines control coefficients for optimal transformation. Multiresolution strategy is applied to register the image from the downsampled low-resolution image to the original high-resolution images. The number of control points also hierarchally increase along the multiresolution framework. The deformation computed at low resolution is the initial transformation for the optimization at the high resolution.
Finite Element Model Based Deformable Registration
For soft tissue, e.g. tumor, registration, we developed a finite element model (FEM)-based deformable registration method. We have applied this FEM registration method to tumor MRI and PET images. In the first step, we applied a rigid registration algorithm to align the cropped MRI and PET images using three translations and three rotations. After registration, we manually segmented the tumor slice-by-slice on both high-resolution MRI and PET image volumes. We then applied the deformable registration algorithm. The registration approach deforms the tumor surface from the MRI volume toward that from the PET image. The displacements at the surface vertices are the force that drives the elastic surface from MRI toward that from the PET image. The tumor was modeled as a linear isotopic elastic material. The FEM model was used to infer volumetric deformation of the tumor from the surface. The force is then integrated over each element and is distributed over the nodes belonging to the element using its shape functions. After obtaining the displacement field for all vertices, we used a linear interpolation to obtain the deformed image volume of the tumor.
Two-Dimensional (2D) Image Registration Software
The 2D registration program can align 2D images using rigid transformation. It integrates manual and automatic registration. It has the following features: (1) Register two 2D images automatically by intensity-based methods, (2) Register two 2D images manually. (3) Load and register floating multiple slices to the reference slice and display the registration parameters for each floating slice.
Three-Dimensional (3D) Image Registration Software
The 3D registration program can align image volumes from CT, PET, MRI, and/or other imaging modalities. It has the following features: (1) Resize the floating volume according to the reference volume size and resolution so as to keep the same volume resolution in every direction for the two volumes; (2) Automatic registration based on mutual information; (3) Manual registration using 3D translation and rotation; (4) It includes two deformable registration approaches; (5) Displays volume in all directions as well as the fusion results; and (6) Displays the location line in 3D space and easily locates each point in 3D space. The user interface is also very friendly and direct.
3D to 2D Registration
This project implements a three-dimensional (3D) to two-dimensional (2D) registration for computed tomography (CT) and dual-energy digital radiography (DR) for the detection of coronary artery calcification. In order to utilize CT as the “gold standard” to evaluate the ability of DR images to detect and localize calcium, we developed an automatic intensity-based 3D-to-2D registration method for 3D CT volumes and 2D DR images. To generate digital rendering radiographs (DRR) from the CT volumes, we developed three projection methods, including Gaussian-weighted projection, threshold-based projection, and average-based projection, were developed. Cross correlation (NCC) and normalized mutual information (NMI) are used as the similarity measurement.
The software program has the following capabilities: (1) Simulate DR images from the reference CT volume at any angle and generate the projection image using Gaussian-weighted projection, threshold-based projection, and average-based projection methods; and (2) Perform registration between the original DR images and the DRR image reconstructed from the CT volume.
Slice to Volume Registration
Slice to volume registration is used to register a two-dimensional image slice to a three-dimensional image volume. In this study, we registered live-time interventional magnetic resonance imaging (iMRI) slices with a previously obtained, high resolution MRI volume which in turn can be registered with a variety of functional images, e.g. PET and SPECT, for tumor targeting. We created and evaluated a slice-to-volume registration algorithm with special features for its potential use in iMRI-guided, radiofrequency (RF) thermal ablation. The algorithm features included a multi-resolution approach, two similarity measures, and automatic restarting in order to avoid local minima. Imaging experiments were performed on volunteers using a conventional diagnostic MR scanner and an interventional MRI system under realistic conditions. Both high-resolution MR volumes and actual iMRI image slices were acquired from the same volunteers. Actual and simulated iMRI images were used to test the dependence of slice-to-volume registration on image noise, coil inhomogeneity, and RF needle artifacts. To quantitatively assess registration, we calculated the mean voxel displacement over a volume of interest between slice-to-volume registration and volume-to-volume registration, which was previously shown to be quite accurate. More than 800 registration experiments were performed. For transverse image slices covering the prostate, the slice-to-volume registration algorithm was 100% successful with an error of < 2 mm, and the average and standard deviation was only 0.4 mm ± 0.2 mm. Visualizations such as combined sector display and contour overlay showed excellent registration of the prostate and other organs throughout the pelvis. Error was greater when an image slice was obtained at other orientations and positions, mostly because of inconsistent image content such as that obtained from variable rectal and bladder filling. These preliminary experiments indicate that MR slice-to-volume registration is sufficiently accurate to be able to aid image-guided therapy. [caption id="attachment_91" align="aligncenter" width="566"] Figure 7. Slice to Volume Registration[/caption]
Zhou X, Ma L, Mubarak HK, Little JV, Chen AY, Myers LL, Sumer BD, Fei B. Automatic detection of head and neck squamous cell carcinoma on pathologic slides using polarized hyperspectral imaging and deep learning. In Medical Imaging 2022: Digital and Computational Pathology 2022 Apr 4 (Vol. 12039, pp. 91-100). SPIE.[PDF]
Pathour T, Akter N, Dormer JD, Chaudhary S, Fei B, Sirsi S. Identifying unique acoustic signatures from chemically-crosslinked microbubble clusters using deep learning. In Medical Imaging 2022: Ultrasonic Imaging and Tomography 2022 Apr 4 (Vol. 12038, pp. 128-136). SPIE.[PDF]
Leitch K, Shahedi M, Dormer JD, Do QN, Xi Y, Lewis MA, Herrera CL, Spong CY, Madhuranthakam AJ, Twickler DM, Fei B. Placenta accreta spectrum and hysterectomy prediction using MRI radiomic features. In Medical Imaging 2022: Computer-Aided Diagnosis 2022 Apr 4 (Vol. 12033, pp. 394-399). SPIE.[PDF]
Leitch K, Halicek M, Shahedi M, Little JV, Chen AY, Fei B. Detecting aggressive papillary thyroid carcinoma using hyperspectral imaging and radiomic features. In Medical Imaging 2022: Computer-Aided Diagnosis 2022 Apr 4 (Vol. 12033, pp. 537-544). SPIE.[PDF]
Huang J, Guo J, Pedrosa I, Fei B. Deep learning-based deformable registration of dynamic contrast enhanced MR images of the kidney. In Medical Imaging 2022: Image-Guided Procedures, Robotic Interventions, and Modeling 2022 Apr 4 (Vol. 12034, pp. 213-222). SPIE.[PDF]
Dormer JD, Villordon M, Shahedi M, Do QN, Xi Y, Lewis MA, Madhuranthakam AJ, Herrera CL, Spong CY, Twickler DM, Fei B. CascadeNet for hysterectomy prediction in pregnant women due to placenta accreta spectrum. In Medical Imaging 2022: Image Processing 2022 Apr 4 (Vol. 12032, pp. 156-164). SPIE.[PDF]
Ma L, Little JV, Chen AY, Myers L, Sumer BD, Fei B. Automatic detection of head and neck squamous cell carcinoma on histologic slides using hyperspectral microscopic imaging. Journal of Biomedical Optics. 2022 Apr;27(4):046501.[PDF]
Ayala L, Isensee F, Wirkert SJ, Vemuri AS, Maier-Hein KH, Fei B, Maier-Hein L. Band selection for oxygenation estimation with multispectral/hyperspectral imaging. Biomedical Optics Express. 2022 Mar 1;13(3):1224-42.[PDF]
Shahedi M, Dormer JD, Halicek M, Fei B. The effect of image annotation with minimal manual interaction for semiautomatic prostate segmentation in CT images using fully convolutional neural networks. Medical physics. 2022 Feb;49(2):1153-60.[PDF]
Nalawade SS, Fang FY, Yogananda CG, Murugesan GK, Shah BR, Pinho MC, Wagner BC, Xi Y, Mickey B, Patel TR, Fei B. Brain tumor IDH, 1p/19q, and MGMT molecular classification using MRI-based deep learning: an initial study on the effect of motion and motion correction. Journal of Medical Imaging. 2022 Jan;9(1):016001.[PDF]
Selim M, Zhang J, Fei B, Zhang GQ, Chen J. Ct image harmonization for enhancing radiomics studies. In2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) 2021 Dec 9 (pp. 1057-1062). IEEE.[PDF]
Selim M, Zhang J, Fei B, Zhang GQ, Ge GY, Chen J. Cross-Vendor CT Image Data Harmonization Using CVH-CT. arXiv preprint arXiv:2110.09693. 2021 Oct 19.[PDF]
Shabestri B, Anastasio MA, Fei B, Leblond F. Special Series Guest Editorial: Artificial Intelligence and Machine Learning in Biomedical Optics. Journal of biomedical optics. 2021 May;26(5).[PDF]
Yogananda CG, Shah BR, Nalawade SS, Murugesan GK, Yu FF, Pinho MC, Wagner BC, Mickey B, Patel TR, Fei B, Madhuranthakam AJ. MRI-based deep-learning method for determining glioma MGMT promoter methylation status. American Journal of Neuroradiology. 2021 May 1;42(5):845-52.[PDF]
Yociss M, Fei B. GPU-based simulation of echocardiography volumes using quantitative fiber-angle-to-backscatter measurements. In Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series 2021 Feb (Vol. 11602, p. 116020U).[PDF]
Wan C, Ma L, Liu X, Fei B. Computer-aided classification of lung nodules on CT images with expert knowledge. In Medical Imaging 2021: Image-Guided Procedures, Robotic Interventions, and Modeling 2021 Feb 15 (Vol. 11598, p. 115982K). International Society for Optics and Photonics.[PDF]
Edwards K, Halicek M, Little JV, Chen AY, Fei B. Multiparametric radiomics for predicting the aggressiveness of papillary thyroid carcinoma using hyperspectral images. In Medical Imaging 2021: Computer-Aided Diagnosis 2021 Feb 15 (Vol. 11597, p. 1159728). International Society for Optics and Photonics.[PDF]
Ma L, Fei B. Comprehensive review of surgical microscopes: technology development and medical applications. Journal of biomedical optics. 2021 Jan;26(1):010901.[PDF]
Ortega S, Halicek M, Fabelo H, Quevedo E, Fei B, Callico GM. Information Extraction Techniques in Hyperspectral Imaging Biomedical Applications. InMultimedia Information Retrieval 2020 Oct 10. IntechOpen.[PDF]
Tran CT, Halicek M, Dormer JD, Tandon A, Hussain T, Fei BW (Corresponding author). Fully automated segmentation of the right ventricle in patients with repaired Tetralogy of Fallot using U-Net. Medical Imaging 2020: Biomedical Applications in Molecular, Structural, and Functional Imaging; 11317(113171M). International Society for Optics and Photonics,[PDF]
Bangalore Yogananda CG, Shah BR, Vejdani-Jahromi M, Nalawade SS, Murugesan GK, Yu FF, Pinho MC, Wagner BC, Mickey B, Patel TR, Fei BW (Corresponding author). A novel fully automated MRI-based deep-learning method for classification of IDH mutation status in brain gliomas. Neuro-oncology; 22(3):402-11.[PDF]
Shahedi M, Halicek M, Dormer JD, Fei BW (Corresponding author). Incorporating minimal user input into deep learning based image segmentation. Medical Imaging 2020: Image Processing; 11313(1131313). International Society for Optics and Photonics,[PDF]
Mavuduru A, Halicek M, Shahedi M, Little JV, Chen AY, Myers LL, Fei BW (Corresponding author). Using a 22-layer U-Net to perform segmentation of squamous cell carcinoma on digitized head and neck histological images. Medical Imaging 2020: Digital Pathology; 11320(113200C). International Society for Optics and Photonics,[PDF]
Shahedi M, Dormer JD, TT AD, Do QN, Xi Y, Lewis MA, Madhuranthakam AJ, Twickler DM, Fei BW (Corresponding author). Segmentation of uterus and placenta in MR images using a fully convolutional neural network. Medical Imaging 2020: Computer-Aided Diagnosis; 11314(113141R). International Society for Optics and Photonics,[PDF]
Hao D, Ding S, Qiu L, Lv Y, Fei BW, Zhu Y, Qin B. Sequential vessel segmentation via deep channel attention network. Neural Networks.[PDF]
Ortega S, Halicek M, Fabelo H, Callico GM, Fei BW (Corresponding author). Hyperspectral and multispectral imaging in digital and computational pathology: a systematic review. Biomedical Optics Express; 11(6): 3195-3233.[PDF]
Fabelo H, Halicek M, Ortega S, Shahedi M, Szolna A, Pineiro JF, Sosa C, O’Shanahan AJ, Bisshopp S, Espino C, Marquez M, Hernandez M, Carrera D, Morera J, Callicó GM, Sarmiento R, Fei BW (Corresponding author). Deep learning-based framework for in-vivo identification of glioblastoma tumor using hyperspectral images of human brain. Sensors;19(4).[PDF]
Halicek M, Shahedi M, Little JV, Chen AY, Myers LL, Sumer BD, Fei BW, Tomaszewski JE, Ward AD. Detection of squamous cell carcinoma in digitized histological images from the head and neck using convolution neural networks. Proceedings of SPIE Medical Imaging 2019: Digital Pathology.[PDF]
Yogananda CGB, Nalawade SS, Murugesan GK, Wagner B, Pinho MC, Fei BW, Madhuranthakam AJ, Maldjian JA. Fully Automated Brain Tumor Segmentation and Survival Prediction of Gliomas using Deep Learning and MRI. bioRxiv (2019):760157.[PDF]
Lu G, Wang D, Qin X, Muller S, Little JV, Wang X, Chen AY, Chen G, Fei BW (Corresponding author). Histopathology feature mining and association with hyperspectral imaging for the detection of squamous neoplasia. Scientific reports; 9(1): 1-13.[PDF]
Halicek M, Dormer J, Little JV, Chen AY, Myers L, Sumer BD, Fei BW (Corresponding author). Hyperspectral Imaging of Head and Neck Squamous Cell Carcinoma for Cancer Margin Detection in Surgical Specimens from 102 Patients Using Deep Learning. Cancers (2019);11(9):1367.[PDF]
Halicek M, Fabelo H, Ortega S, Callicó GM, Fei BW (Corresponding author). In-vivo and ex-vivo tissue analysis through hyperspectral imaging techniques: Revealing the invisible featuers of cancer. Cancers;11(6).[PDF]
Shahedi M, Halicek M, Guo R, Zhang G, Schuster DM, Fei BW (Corresponding author). A semiautomatic prostate segmentation in CT images using a deep learning approach. The Annual Meeting of the Biomedical Engineering Society.
Tian Z, Liu L, Fei BW. Deep convolutional neural network for prostate MR segmentation. International Journal of Computer Assisted Radiology and Surgery;13(11):1687.[PDF]
Lu G, Wang DS, Qin X, Muller S, Wang X, Chen AY, Chen ZG, Fei BW (Corresponding author). Detection and delineation of squamous neoplasia with hyperspectral imaging in a mouse model of tongue carcinogenesis. Journal of Biophotonics;11(3):e201700078.[PDF]
Tian Z, Liu L, Zhang Z, Fei BW (Corresponding author). PSNet: prostate segmentation on MRI based on a convolutional neural network. Journal of Medical Imaging;5(2):021208.[PDF]
Sarno, A., Dance, DR., van Engen, RE., Young, KC., Russo, P., Di Lillo, F., Mettivier, G., Bliznakova, K., Fei, BW., Sechopoulos, I.(2017). “A Monte Carlo model for mean glandular dose evaluation in spot compression mammography.”Medical Physics 44(7): 3848-3860.[PDF] [DOI]
Ma L, Guo R, Zhang G, Tade F, Schuster DM, Fei BW, Automatic segmentation of the prostate on CT images using deep learning and multi-atlas fusion, Proc. SPIE 10133, Medical Imaging 2017: Image Processing, 101332O, February 24, 2017, Orlando, FL.[PDF]
Fei BW, Lu G, Wang X, Zhang HZ, Little JV, Patel MR, Griffith CC, El-Diery MW, Chen AY. Label-free reflectance hyperspectral imaging for tumor margin assessment: a pilot study on surgical specimens of cancer patients. Journal of Biomedical Optics;22(8):086009.[PDF]
Chung H, Lu G, Tian Z, Wang D, Chen ZG, Fei BW (Corresponding author). Superpixel-based spectral classification for the detection of head and neck cancer with hyperspectral imaging. Proceedings of SPIE – The International Society for Optical Engineering 2016; 9788:978813. PubMed PMID:27656035.[PDF]
Pike R, Lu G, Wang D, Chen ZG, Fei BW (Corresponding author). A minimum spanning forest based classification method for dedicated breast CT images. Medical Physics 2015; 42:6190-202.[PDF]
Pike R, Sechopoulos I, Fei BW (Corresponding author). A minimum spanning forest-based method for noninvasive cancer detection wtih hyperspectral imaging. IEEE Transactions on Biomedical Engineering 2015; 63(3):653-663.[PDF]
Qin X, Wang S, Shen M, Zhang X, Lerakis S, Wagner MB, Fei BW (Corresponding author). Register cardiac fiber orientations from 3D DTI volume to 2D ultrasound image of rat hearts. Proceedings of SPIE – The International Society for Optical Engineering 2015; 9415:94152M. PubMed PMID:26855466.[PDF]
Lu G, Halig L, Wang D, Chen ZG, Fei BW (Corresponding author). Spectral-spatial classification using tensor modeling for cancer detection with hyperspectral imaging. Proceedings of SPIE – The International Society for Optical Engineering 2014; 9034:903413. PubMed PMID:25328639.[PDF]
Akbari H, Fei BW (Corresponding author). Automatic 3D segmentation of the kidney in MR images using wavelet feature extraction and probability shape model. Proceedings of SPIE – The International Society for Optical Engineering 2013; 8314:83143D. PubMed PMID:24027620.[PDF]
Halig LV, Wang D, Wang AY, Chen ZG, Fei BW (Corresponding author).Biodistribution study of nanoparticle encapsulated photodynamic therapy drugs using multispectral imaging. Proceedings of SPIE – The International Society for Optical Engineering 2013; 8672. PubMed PMID:24236230.[PDF]
Yang X, Fei BW (Corresponding author).3D prostate segmentation of ultrasound images combining longitudinal image registration and machine learning. Proceedings of SPIE – The International Society for Optical Engineering 2012; 8316:83162O. PubMed PMID:24027622.[PDF]
Wang H, Fei BW (Corresponding author). An MR image-guided, voxel-based partial volume correction method for PET images. Medical Physics 2012; 39:179-195.[PDF]
Akbari H, Halig LV, Schuster DM, Osunkoya A, Master VA, Nieh PT, Chen GZ, and Fei BW (Corresponding author). Hyperspectral imaging and quantitative analysis for prostate cancer detection, Journal of Biomedical Optics 2012; 17:076005.[PDF]
Yang X, Akbari H, Halig L, Fei BW (Corresponding author). 3D non-rigid registration using surface and local salient features for transrectal ultrasound image-guided prostate biopsy. Proceedings of SPIE – The International Society for Optical Engineering 2011; 7964:79642V. PubMed PMID:24027609.[PDF]
Yang X, Schuster D, Master V, Nieh P, Fenster A, Fei BW (Corresponding author). Automatic 3D segmentation of ultrasound images using atlas registration and statistical texture prior. Proceedings of SPIE – The International Society for Optical Engineering 2011; 7964:796432. PubMed PMID:22708024. (Cum Laude Poster Award – First Place).[PDF]
Fei BW, Yang X, Nye JA, Aarsvold JN, Meltzer CC, Votaw JR. PET-MRI quantification tools – registration, segmentation, classification, and attenuation correction. IEEE Nuclear Science Symposium and Medical Imaging Conference Focused Workshop on PET/MRI, November 1, 2010, Knoxville, TN.
Wang H, Feyes D, Mulvihill J, Oleinick N, Maclennan G, Fei BW (Corresponding author). Multiscale fuzzy C-means image classification for multiple weighted MR images for the assessment of photodynamic therapy in mice. Proceedings of SPIE – The International Society for Optical Engineering 2007; 6512. PubMed PMID:24386526.[PDF]
Haaga JR, Exner A, Fei BW, Seftel A. Semiquantitative imaging measurement of baseline and vasomodulated normal prostatic blood flow using sildenafil. International Journal of Impotence Research 2007; 19:110-3.[PDF]
Fei BW (Corresponding author), Chen X, Wang H, Sabol JM, DuPont E, Gilkeson RC. Automatic registration of CT volumes and dual-energy digital radiography for detection of cardiac and lung diseases. Proceedings of IEEE Engineering in Medicine and Biology Society 2006; 1:1976-9. PubMed PMID:17945687.[PDF]
Fei BW (Corresponding author), Flask C, Wang H, Pi A, Wilson D, Shillingford J, Murcia N, Weimbs T, Duerk J. Image segmentation, registration and visualization of serial MR images for therapeutic assessment of polycystic kidney disease in transgenic mice. Proceedings of IEEE Engineering in Medicine and Biology Society 2005; 1:467-9. PubMed PMID:17282217.[PDF]
Fei BW, Zhuang T. Computer-assisted surgery and its localization methods. Foreign Medical Science 1997; 20:199-204.