This PDF file contains the front matter associated with SPIE Proceedings Volume 10579, including the Title Page, Copyright information, Table of Contents, and Conference Committee listing.

Conference Presentation for "Will computers replace radiologists for primary reads in 20 years: a debate"

Purpose: PACS, digital report and informatics were originally developed for radiology application to replace film and paper. During the past more thirty more years, these technologies have been integrated and progressed to a must radiological diagnosis clinical component for daily use. Profession Huang wrote two Editions of the “PACS and Imaging Informatics” series in 2004 and 2010 published by Wiley and Sons. Due to the maturity of PACS and major advancement in imaging informatics during the past seven years, the original DICOM PACS and informatics technologies has been extended and expanded to many related clinical imaging applications. However, adopting DICOM standard to be a general image based standard in Internet communications for other types of non-radiological images and signals has some major drawbacks. There were many new protocols and technologies being introduced into PACS and Imaging Informatics practices and industry in past few years. This presentation is brief of new “PACS and Imaging Informatics” series to reflect the new developments, but the name of the series has been slightly changed to “PACS-BASED MULTIMEDIA IMAGING INFORMATICS”, Third Edition, which will be published by Wiley and Sons in 2018. Major Content and Materials of the Third Edition: This series was written to discuss the development and growth of medical imaging and PACS related technology during the past 25 more years. This presentation is the continuation of the “PACS and Imaging Informatics” series in 2004 and 2010 published by Wiley and Sons, and being organized in four Parts in 22 Chapters: Part I. The Beginning – Retrospective Part II: Medical Imaging, PACS fundamental, Industrial Guidelines, Standards, and Compliance Part III: Informatics, Data Grid, Workstation, Radiation Therapy, Simulators, Molecular Imaging, Archive Server and Cloud Computing Part IV: Multimedia Imaging Informatics, Computer-Aided Diagnosis (CAD), Image-Guided Decision Support, Proton Therapy, Minimally Invasive Multimedia Image-Assisted Surgery, Big Data. Selected portions of the PACS-based Imaging Informatics series 2004, 2010 and some pre-printed materials in 2017 have been used as lecture materials in undergraduate and graduate courses: "Medical Imaging and Advanced Instrumentation" at UCLA, UCSF, and UC Berkeley; "Biomedical Engineering Lectures" in Taiwan, and the People's Republic of China; "PACS and Medical Imaging Informatics" at the Hong Kong Polytechnic University; and required courses at the "Medical Imaging and Informatics" track at the Department of Biomedical Engineering, School of Engineering, USC. The Future Growth: The term “PACS-based imaging multimedia” can loosely mean that any clinical specialist would have the opportunity to extract and integrate the patient’s existing data, images, graphs, tables, reports, 3-D, 4-D, 5-D images, movies, and scripts based on the PACS-based technology to compose their needed contents in a physician workstation, as well as to view, diagnose, report, and archive. Multimedia can be recorded, played, displayed dynamically and interactively accessed from information content processing devices such as computerized and electronic device, and can as well as be part of a live performance. The majority practicing physicians nowadays do have experience in using PACS-based images and reports from medical Web servers and workstations to help them to take care of their patients. Using the PACS-based technology platform, these multimedia PACS-based biomedical imaging informatics can enrich the clinical specialists to facilitate their patient care. It is our greatest hope that this new edition will continuously be used not only to provide information and guidelines for those contemplating a PACS-based Imaging Informatics career but also inspire others to apply this technology as a tool toward a brighter future for healthcare delivery. Some Key Protocols and Technologies Discussed in the Third Edition: In early 2010s, RSNA (Radiological Society of North America) along with the Integrating Healthcare Enterprise (IHE) initiated the concept of “IHE XDS-I profile (Integrating Healthcare Cross-Enterprise Document Sharing - Imaging)”. Several major manufacturers and research Institutes were invited by RSNA to participate in this initiative. Because of the successful of IHE XDS-I profile since 2010, the PACS climate has changed dramatically. It seems that the future trend of PACS is broken into pieces, the term is now called “deconstructed PACS”. Traditional PACS vendors are now not only focusing their business in pure PACS market or storage archive solution, but also on viewing software and imaging workstations. And storage solution is now called the VNA technology (Vendor Neutral Archive) that concentrates storing the image files in the native DICOM file. VNA is also used to store all other kinds of data including non-Radiology images whether they are DICOM or non-DICOM. PACS vendors still sell PACS because they have knowledge of the Radiology workflow so that their viewing software will embed these workflow features. Most VNA technologies don’t know the deeper DICOM fields, they just extract basic patient information and archive by using the Web technology and the IHE XDS-I profile. The recent drastic change of “PACS” concept has inspired and influenced the further understanding of the combination of DICOM, Medical imaging, PACS and Informatics.

Prostate cancer is the most common non-skin related cancer affecting 1 in 7 men in the United States. Treatment of patients with prostate cancer still remains a difficult decision-making process that requires physicians to balance clinical benefits, life expectancy, comorbidities, and treatment-related side effects. Gleason score (a sum of the primary and secondary Gleason patterns) solely based on morphological prostate glandular architecture has shown as one of the best predictors of prostate cancer outcome. Significant progress has been made on molecular subtyping prostate cancer delineated through the increasing use of gene sequencing. Prostate cancer patients with Gleason score of 7 show heterogeneity in recurrence and survival outcomes. Therefore, we propose to assess the correlation between histopathology images and genomic data with disease recurrence in prostate tumors with a Gleason 7 score to identify prognostic markers. In the study, we identify image biomarkers within tissue WSIs by modeling the spatial relationship from automatically created patches as a sequence within WSI by adopting a recurrence network model, namely long short-term memory (LSTM). Our preliminary results demonstrate that integrating image biomarkers from CNN with LSTM and genomic pathway scores, is more strongly correlated with patients recurrence of disease compared to standard clinical markers and engineered image texture features. The study further demonstrates that prostate cancer patients with Gleason score of 4+3 have a higher risk of disease progression and recurrence compared to prostate cancer patients with Gleason score of 3+4.

Investigating the association between brain regions and genes continues to be a challenging topic in imaging genetics. Current brain region of interest (ROI)-gene association studies normally reduce data dimension by averaging the value of voxels in each ROI. This averaging may lead to a loss of information due to the existence of functional sub-regions. Pearson correlation is widely used for association analysis. However, it only detects linear correlation whereas nonlinear correlation may exist among ROIs. In this work, we introduced distance correlation to ROI-gene association analysis, which can detect both linear and nonlinear correlations and overcome the limitation of averaging operations by taking advantage of the information at each voxel. Nevertheless, distance correlation usually has a much lower value than Pearson correlation. To address this problem, we proposed a hybrid correlation analysis approach, by applying canonical correlation analysis (CCA) to the distance covariance matrix instead of directly computing distance correlation. Incorporating CCA into distance correlation approach may be more suitable for complex disease study because it can detect highly associated pairs of ROI and gene groups, and may improve the distance correlation level and statistical power. In addition, we developed a novel nonlinear CCA, called distance kernel CCA, which seeks the optimal combination of features with the most significant dependence. This approach was applied to imaging genetic data from the Philadelphia Neurodevelopmental Cohort (PNC). Experiments showed that our hybrid approach produced more consistent results than conventional CCA across resampling and both the correlation and statistical significance were increased compared to distance correlation analysis. Further gene enrichment analysis and region of interest (ROI) analysis confirmed the associations of the identified genes with brain ROIs. Therefore, our approach provides a powerful tool for finding the correlation between brain imaging and genomic data.

Breast cancer is a highly heterogeneous disease both biologically and clinically, and certain pathologic parameters, i.e., Ki67 expression, are useful in predicting the prognosis of patients. The aim of the study is to identify intratumor heterogeneity of breast cancer for predicting Ki-67 proliferation status in estrogen receptor (ER)-positive breast cancer patients. A dataset of 77 patients was collected who underwent dynamic contrast enhancement magnetic resonance imaging (DCE-MRI) examination. Of these patients, 51 were high-Ki-67 expression and 26 were low-Ki-67 expression. We partitioned the breast tumor into subregions using two methods based on the values of time to peak (TTP) and peak enhancement rate (PER). Within each tumor subregion, image features were extracted including statistical and morphological features from DCE-MRI. The classification models were applied on each region separately to assess whether the classifiers based on features extracted from various subregions features could have different performance for prediction. An area under a receiver operating characteristic curve (AUC) was computed using leave-one-out cross-validation (LOOCV) method. The classifier using features related with moderate time to peak achieved best performance with AUC of 0.826 than that based on the other regions. While using multi-classifier fusion method, the AUC value was significantly (P=0.03) increased to 0.858±0.032 compare to classifier with AUC of 0.778 using features from the entire tumor. The results demonstrated that features reflect heterogeneity in intratumoral subregions can improve the classifier performance to predict the Ki-67 proliferation status than the classifier using features from entire tumor alone.

Breast cancer, with its high heterogeneity, is the most common malignancies in women. In addition to the entire tumor itself, tumor microenvironment could also play a fundamental role on the occurrence and development of tumors. The aim of this study is to investigate the role of heterogeneity within a tumor and the surrounding stromal tissue in predicting the Ki-67 proliferation status of oestrogen receptor (ER)-positive breast cancer patients. To this end, we collected 62 patients imaged with preoperative dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) for analysis. The tumor and the peritumoral stromal tissue were segmented into 8 shells with 5 mm width outside of tumor. The mean enhancement rate in the stromal shells showed a decreasing order if their distances to the tumor increase. Statistical and texture features were extracted from the tumor and the surrounding stromal bands, and multivariate logistic regression classifiers were trained and tested based on these features. An area under the receiver operating characteristic curve (AUC) were calculated to evaluate performance of the classifiers. Furthermore, the statistical model using features extracted from boundary shell next to the tumor produced AUC of 0.796±0.076, which is better than that using features from the other subregions. Furthermore, the prediction model using 7 features from the entire tumor produced an AUC value of 0.855±0.065. The classifier based on 9 selected features extracted from peritumoral stromal region showed an AUC value of 0.870±0.050. Finally, after fusion of the predictive model obtained from entire tumor and the peritumoral stromal regions, the classifier performance was significantly improved with AUC of 0.920. The results indicated that heterogeneity in tumor boundary and peritumoral stromal region could be valuable in predicting the indicator associated with prognosis.

Nasal septal perforations (NSPs) are relatively common. They can be problematic for both patients and head and neck reconstructive surgeons who attempt to repair them. Often, this repair is made using an interpositional graft sandwiched between bilateral mucoperichondrial advancement flaps. The ideal graft is nasal septal cartilage. However, many patients with NSP lack sufficient septal cartilage to harvest. Harvesting other sources of autologous cartilage grafts, such as auricular cartilage, adds morbidity to the surgical case and results in a graft that lacks the ideal qualities required to repair the nasal septum. Tissue engineering has allowed for new reconstructive protocols to be developed. Currently, the authors are unaware of any new literature that looks to improve repair of NSP using custom tissue-engineered cartilage grafts. The first step of this process involves developing a protocol to print the graft from a patient's pre-operative CT.

In this study, CT scans were converted into STereoLithography (STL) file format. The subsequent STL files were transformed into 3D printable G-Code using the Slic3r software. This allowed us to customize the parameters of our print and we were able to choose a layer thickness of 0.1mm. A desktop 3D bioprinter (BioBot 1) was then used to construct the scaffold.

This method resulted in the production of a PCL scaffold that precisely matched the patient’s nasal septal defect, in both size and shape. This serves as the first step in our goal to create patient-specific tissue engineered nasal septal cartilage grafts for NSP repair.

In this presentation, we presented a new approach to design cloud-based image sharing network for collaborative imaging diagnosis and consultation through Internet, which can enable radiologists, specialists and physicians locating in different sites collaboratively and interactively to do imaging diagnosis or consultation for difficult or emergency cases. The designed network combined a regional RIS, grid-based image distribution management, an integrated video conferencing system and multi-platform interactive image display devices together with secured messaging and data communication. There are three kinds of components in the network: edge server, grid-based imaging documents registry and repository, and multi-platform display devices. This network has been deployed in a public cloud platform of Alibaba through Internet since March 2017 and used for small lung nodule or early staging lung cancer diagnosis services between Radiology departments of Huadong hospital in Shanghai and the First Hospital of Jiaxing in Zhejiang Province.

When processing large medical imaging studies, adopting high performance grid computing resources rapidly becomes important. We recently presented a "medical image processing-as-a-service" grid framework that offers promise in utilizing the Apache Hadoop ecosystem and HBase for data colocation by moving computation close to medical image storage. However, the framework has not yet proven to be easy to use in a heterogeneous hardware environment. Furthermore, the system has not yet validated when considering variety of multi-level analysis in medical imaging. Our target design criteria are (1) improving the framework’s performance in a heterogeneous cluster, (2) performing population based summary statistics on large datasets, and (3) introducing a table design scheme for rapid NoSQL query. In this paper, we present a heuristic backend interface application program interface (API) design for Hadoop and HBase for Medical Image Processing (HadoopBase-MIP). The API includes: Upload, Retrieve, Remove, Load balancer (for heterogeneous cluster) and MapReduce templates. A dataset summary statistic model is discussed and implemented by MapReduce paradigm. We introduce a HBase table scheme for fast data query to better utilize the MapReduce model. Briefly, 5153 T1 images were retrieved from a university secure, shared web database and used to empirically access an in-house grid with 224 heterogeneous CPU cores. Three empirical experiments results are presented and discussed: (1) load balancer wall-time improvement of 1.5-fold compared with a framework with built-in data allocation strategy, (2) a summary statistic model is empirically verified on grid framework and is compared with the cluster when deployed with a standard Sun Grid Engine (SGE), which reduces 8-fold of wall clock time and 14-fold of resource time, and (3) the proposed HBase table scheme improves MapReduce computation with 7 fold reduction of wall time compare with a naïve scheme when datasets are relative small. The source code and interfaces have been made publicly available.

Glaucoma neuropathy is a major cause of irreversible blindness worldwide. Current models of chronic care will not be able to close the gap of growing prevalence of glaucoma and challenges for access to healthcare services. Teleophthalmology is being developed to close this gap. In order to develop automated techniques for glaucoma detection which can be used in tele-ophthalmology we have developed a large retinal fundus dataset. A de-identified dataset of retinal fundus images for glaucoma analysis (RIGA) was derived from three sources for a total of 750 images. The optic cup and disc boundaries for each image was marked and annotated manually by six experienced ophthalmologists and included the cup to disc (CDR) estimates. Six parameters were extracted and assessed (the disc area and centroid, cup area and centroid, horizontal and vertical cup to disc ratios) among the ophthalmologists. The inter-observer annotations were compared by calculating the standard deviation (SD) for every image between the six ophthalmologists in order to determine if the outliers amongst the six and was used to filter the corresponding images. The data set will be made available to the research community in order to crowd source other analysis from other research groups in order to develop, validate and implement analysis algorithms appropriate for tele-glaucoma assessment. The RIGA dataset can be freely accessed online through University of Michigan, Deep Blue website (doi:10.7302/Z23R0R29).

An emerging trend in AD research is brain network development including graphic metrics and graph mining techniques. To construct a brain structural network, Diffusion Tensor Imaging (DTI) in conjunction with T1 weighted Magnetic Resonance Imaging (MRI) can be used to isolate brain regions as nodes, white matter tracts as the edge, and the density of the tracts as the weight to the edge. To study such network, its sub-network is often obtained by excluding unrelated nodes or edges. Existing research has heavily relied on domain knowledge or single-thresholding individual subject based network metrics to identify the sub network. In this research, we develop a bi-threshold frequent subgraph mining method (BT-FSG) to automatically filter out less important edges in responding to the clinical questions. Using this method, we are able to discover a subgraph of human brain network that can significantly reveal the difference between cognitively unimpaired APOE-4 carriers and noncarriers based on the correlations between the age vs. network local metric and age vs. network or global metric. This can potentially become a brain network marker for evaluating the AD risks for preclinical individuals.

In this paper, we present a method for automatically identifying the gender of an imaged person using their frontal chest x-ray images. Our work is motivated by the need to determine missing gender information in some datasets. The proposed method employs the technique of convolutional neural network (CNN) based deep learning and transfer learning to overcome the challenge of developing handcrafted features in limited data. Specifically, the method consists of four main steps: pre-processing, CNN feature extractor, feature selection, and classifier. The method is tested on a combined dataset obtained from several sources with varying acquisition quality resulting in different pre-processing steps that are applied for each. For feature extraction, we tested and compared four CNN architectures, viz., AlexNet, VggNet, GoogLeNet, and ResNet. We applied a feature selection technique, since the feature length is larger than the number of images. Two popular classifiers: SVM and Random Forest, are used and compared. We evaluated the classification performance by cross-validation and used seven performance measures. The best performer is the VggNet-16 feature extractor with the SVM classifier, with accuracy of 86.6% and ROC Area being 0.932 for 5-fold cross validation. We also discuss several misclassified cases and describe future work for performance improvement.

Reviewing interval cancers and prior screening mammograms are a key measure to monitor screening performance. Radiological analysis of the imaging features in prior mammograms and retrospective classification are an important educational tool for readers to improve individual performance.

The requirements of remote, collaborative image review sessions, such as those required to run a remote interval cancer review, are variable and demand a flexible and configurable software solution that is not currently available on commercial workstations. The wide range of requirements for both collection and remote review of interval cancers has precipitated the creation of extensible medical image viewers and accompanying systems.

In order to allow remote viewing, an application has been designed to allow workstation-independent, PACS-less viewing and interaction with medical images in a remote, collaborative manner, providing centralised reporting and web-based feedback. A semi-automated process, which allows the centralisation of interval cancer cases, has been developed. This stand-alone, flexible image collection toolkit provides the extremely important function of bespoke, ad-hoc image collection at sites where there is no dedicated hardware.

Web interfaces have been created which allow a national or regional administrator to organise, coordinate and administer interval cancer review sessions and deploy invites to session members to participate. The same interface allows feedback to be analysed and distributed.

The eICR provides a uniform process for classifying interval cancers across the NHSBSP, which facilitates rapid access to a robust 'external' review for patients and their relatives seeking answers about why their cancer was 'missed'.

In this paper, we study low-cost motion tracking systems for range of motion (RoM) measurements in the tele-rehabilitation context using Augmented Reality. We propose simple yet effective extensions of the Microsoft Kinect SDK 2.0 skeleton tracking algorithm. Our extensions consist of temporal smoothing of the joint estimates as well as an intuitive, patient-specific adjustment of the bone lengths that is implemented as a quick, one-time calibration performed by the therapist. We compare our system to the Kinect v1, the non-modified Kinect v2, a marker-based optical tracking system, and the clinical gold standard set by two subject-matter-experts using a goniometer. We study the accuracy of all systems in RoM measurement on the elbow joints. We quantitatively compare angular deviation from the expert measurements and perform analysis on statistical confidence. The results indicate, that the proposed personalized setup substantially outperforms all competing systems and effectively corrects for the systematic error of the skeleton tracking, particularly at full flexion. The improved system matched the observations of both experts with a mean error of 3:78° We conclude, that the proposed, personalized method for RoM measurement with Augmented Reality feedback is promising for tele-rehabilitation scenarios. Future work will investigate whether similar strategies can be applied to more complex joints, such as the shoulder.

Bioprinting of tissue has its applications throughout medicine. Recent advances in medical imaging allows the generation of 3-dimensional models that can then be 3D printed. However, the conventional method of converting medical images to 3D printable G-Code instructions has several limitations, namely significant processing time for large, high resolution images, and the loss of microstructural surface information from surface resolution and subsequent reslicing. We have overcome these issues by creating a JAVA program that skips the intermediate triangularization and reslicing steps and directly converts binary dicom images into G-Code.

In this study, we tested the two methods of G-Code generation on the application of synthetic bone graft scaffold generation. We imaged human cadaveric proximal femurs at an isotropic resolution of 0.03mm using a high resolution peripheral quantitative computed tomography (HR-pQCT) scanner. These images, of the Digital Imaging and Communications in Medicine (DICOM) format, were then processed through two methods. In each method, slices and regions of print were selected, filtered to generate a smoothed image, and thresholded. In the conventional method, these processed images are converted to the STereoLithography (STL) format and then resliced to generate G-Code. In the new, direct method, these processed images are run through our JAVA program and directly converted to G-Code. File size, processing time, and print time were measured for each.

We found that this new method produced a significant reduction in G-Code file size as well as processing time (92.23% reduction). This allows for more rapid 3D printing from medical images.

Real-time visualization of 3D medical data on low-performance mobile and virtual reality (VR, e.g. HTC Vive) devices is non-trivial because it is necessary to render image twice per frame for each of the eyes. The algorithm presented in this paper describes an approach that allows visualizing 3D medical data in real-time without loss of quality as well as demanding less computational resources. The proposed method is a two-pass rendering algorithm, whereby the approximate texture is rendered at the first step and optimized detailed ray casting applied to the whole scene at the second step. Since the algorithm requires no preprocessing and both passes are performed on each visualization frame, the algorithm allows to dynamically change the level of rendered isosurfaces; this is one of the chief advantages of the proposed approach. The versatility of the solution allowed its implementation for medical data visualization on various platforms, i.e. HTC Vive, Web Browsers, Android devices, and iOS devices.

Pre-operative CT data is available for several orthopedic and trauma interventions, and is mainly used to identify injuries and plan the surgical procedure. In this work we propose an intuitive augmented reality environment allowing visualization of pre-operative data during the intervention, with an overlay of the optical information from the surgical site. The pre-operative CT volume is first registered to the patient by acquiring a single C-arm X-ray image and using 3D/2D intensity-based registration. Next, we use an RGBD sensor on the C-arm to fuse the optical information of the surgical site with patient pre-operative medical data and provide an augmented reality environment. The 3D/2D registration of the pre- and intra-operative data allows us to maintain a correct visualization each time the C-arm is repositioned or the patient moves. An overall mean target registration error (mTRE) and standard deviation of 5.24 ± 3.09 mm was measured averaged over 19 C-arm poses. The proposed solution enables the surgeon to visualize pre-operative data overlaid with information from the surgical site (e.g. surgeon’s hands, surgical tools, etc.) for any C-arm pose, and negates issues of line-of-sight and long setup times, which are present in commercially available systems.

Patient-specific 3D printed phantoms (3DP) can reproduce accurate patient geometry and provide precise tools for Endovascular Image Guided Interventions (EIGI) simulations. We propose to build and test 3DP phantoms which mimic the arterial wall elasticity and surface properties and demonstrate their utility in comprehensive EIGI simulations.

3DP idealized and patient specific vascular phantoms were manufactured using Stratasys Objet 500 Connex 3. The idealized phantoms were created using a sine wave shape, patient specific phantoms were based on CT- angiography volumes. The phantoms were coated with a hydrophilic material to mimic vascular surface properties. We tested various endovascular procedures using an Interventional Device Testing Equipment (IDTE) 2000 and measured push/pull force used to actuate endovascular devices during EIGIs.

The force needed to advance devices in neurovascular phantoms varied based on tortuosity, material and coating, ranging from -3 to 21 grams-force. Hydrophilic coating reduced maximum force from 21 to 4.8 grams-force in the same model. IDTE 2000 results of neurovascular models were compared to hand manipulation of guidewire access using a six-axis force sensor with forces ranging from -50 to 440 grams. The clot retriever tested in carotid models experienced most friction around tortuous bends ranging from -65 to -90 grams-force, with increasing rigidity of materials creating increased friction. Sine wave model forces varied from -2 to 105 grams.

3DP allows manufacturing of vascular phantoms with precise mechanical and surface properties which can be used for EIGI simulations for imaging protocol optimization and device behavior assessment.

Virtual colonoscopy (VC) allows a radiologist to navigate through a 3D colon model reconstructed from a computed tomography scan of the abdomen, looking for polyps, the precursors of colon cancer. Polyps are seen as protrusions on the colon wall and haustral folds, visible in the VC y-through videos. A complete review of the colon surface requires full navigation from the rectum to the cecum in antegrade and retrograde directions, which is a tedious task that takes an average of 30 minutes. Crowdsourcing is a technique for non-expert users to perform certain tasks, such as image or video annotation. In this work, we use crowdsourcing for the examination of complete VC y-through videos for polyp annotation by non-experts. The motivation for this is to potentially help the radiologist reach a diagnosis in a shorter period of time, and provide a stronger confirmation of the eventual diagnosis. The crowdsourcing interface includes an interactive tool for the crowd to annotate suspected polyps in the video with an enclosing box. Using our work flow, we achieve an overall polyps-per-patient sensitivity of 87.88% (95.65% for polyps ≥5mm and 70% for polyps <5mm). We also demonstrate the efficacy and effectiveness of a non-expert user in detecting and annotating polyps and discuss their possibility in aiding radiologists in VC examinations.

Adolescence is a transitional period between childhood and adulthood with physical changes, as well as increasing emotional activity. Studies have shown that the emotional sensitivity is related to a second dramatical brain growth. However, there is little focus on the trend of brain development during this period. In this paper, we aim to track the functional brain connectivity development in adolescence using resting state fMRI (rs-fMRI), which amounts to a time-series analysis problem. Most existing methods either require the time point to be fairly long or are only applicable to small graphs. To this end, we adapted a fast Bayesian integrative analysis (FBIA) to address the short time-series difficulty, and combined with adaptive sum of powered score (aSPU) test for group difference. The data we used are the resting state fMRI (rs-fMRI) obtained from the publicly available Philadelphia Neurodevelopmental Cohort (PNC). They include 861 individuals aged 8–22 years who were divided into five different adolescent stages. We summarized the networks with global measurements: segregation and integration, and provided full brain functional connectivity pattern in various stages of adolescence. Moreover, our research revealed several brain functional modules development trends. Our results are shown to be both statistically and biologically significant.

Breast cancer can be classified into four molecular subtypes of Luminal A, Luminal B, HER2 and Basal-like, which have significant differences in treatment and survival outcomes. We in this study aim to predict immunohistochemistry (IHC) determined molecular subtypes of breast cancer using image features derived from tumor and peritumoral stroma region based on diffusion weighted imaging (DWI). A dataset of 126 breast cancer patients were collected who underwent preoperative breast MRI with a 3T scanner. The apparent diffusion coefficients (ADCs) were recorded from DWI, and breast image was segmented into regions comprising the tumor and the surrounding stromal. Statistical characteristics in various breast tumor and peritumoral regions were computed, including mean, minimum, maximum, variance, interquartile range, range, skewness, and kurtosis of ADC values. Additionally, the difference of features between each two regions were also calculated. The univariate logistic based classifier was performed for evaluating the performance of the individual features for discriminating subtypes. For multi-class classification, multivariate logistic regression model was trained and validated. The results showed that the tumor boundary and proximal peritumoral stroma region derived features have a higher performance in classification compared to that of the other regions. Furthermore, the prediction model using statistical features, difference features and all the features combined from these regions generated AUC values of 0.774, 0.796 and 0.811, respectively. The results in this study indicate that ADC feature in tumor and peritumoral stromal region would be valuable for estimating the molecular subtype in breast cancer.

Retinopathy of prematurity (ROP) is a disease that affects premature infants, where abnormal growth of the retinal blood vessels can lead to blindness unless treated accordingly. Infants considered at risk of severe ROP are monitored for symptoms of plus disease, characterized by arterial tortuosity and venous dilation at the posterior pole, with a standard photographic definition. Disagreement among ROP experts in diagnosing plus disease has driven the development of computer-based methods that classify images based on hand-crafted features extracted from the vasculature. However, most of these approaches are semi-automated, which are time-consuming and subject to variability. In contrast, deep learning is a fully automated approach that has shown great promise in a wide variety of domains, including medical genetics, informatics and imaging. Convolutional neural networks (CNNs) are deep networks which learn rich representations of disease features that are highly robust to variations in acquisition and image quality. In this study, we utilized a U-Net architecture to perform vessel segmentation and then a GoogLeNet to perform disease classification. The classifier was trained on 3,000 retinal images and validated on an independent test set of patients with different observed progressions and treatments. We show that our fully automated algorithm can be used to monitor the progression of plus disease over multiple patient visits with results that are consistent with the experts’ consensus diagnosis. Future work will aim to further validate the method on larger cohorts of patients to assess its applicability within the clinic as a treatment monitoring tool.

Although breast magnetic resonance imaging (MRI) has been used as a breast cancer screening modality for high-risk women, its cancer detection yield remains low (i.e., ≤ 3%). Thus, increasing breast MRI screening efficacy and cancer detection yield is an important clinical issue in breast cancer screening. In this study, we investigated association between the background parenchymal enhancement (BPE) of breast MRI and the change of diagnostic (BIRADS) status in the next subsequent breast MRI screening. A dataset with 65 breast MRI screening cases was retrospectively assembled. All cases were rated BIRADS-2 (benign findings). In the subsequent screening, 4 cases were malignant (BIRADS-6), 48 remained BIRADS-2 and 13 were downgraded to negative (BIRADS-1). A computer-aided detection scheme was applied to process images of the first set of breast MRI screening. Total of 33 features were computed including texture feature and global BPE features. Texture features were computed from either a gray-level co-occurrence matrix or a gray level run length matrix. Ten global BPE features were also initially computed from two breast regions and bilateral difference between the left and right breasts. Box-plot based analysis shows positive association between texture features and BIRADS rating levels in the second screening. Furthermore, a logistic regression model was built using optimal features selected by a CFS based feature selection method. Using a leave-one-case-out based cross-validation method, classification yielded an overall 75% accuracy in predicting the improvement (or downgrade) of diagnostic status (to BIRAD-1) in the subsequent breast MRI screening. This study demonstrated potential of developing a new quantitative imaging marker to predict diagnostic status change in the short-term, which may help eliminate a high fraction of unnecessary repeated breast MRI screenings and increase the cancer detection yield.

In this work, we present the use of Bayesian networks for radiologist decision support during clinical interpretation. This computational approach has the advantage of avoiding incorrect diagnoses that result from known human cognitive biases such as anchoring bias, framing effect, availability bias, and premature closure. To integrate Bayesian networks into clinical practice, we developed an open-source web application that provides diagnostic support for a variety of radiology disease entities (e.g., basal ganglia diseases, bone lesions). The Clinical tool presents the user with a set of buttons representing clinical and imaging features of interest. These buttons are used to set the value for each observed feature. As features are identified, the conditional probabilities for each possible diagnosis are updated in real time. Additionally, using sensitivity analysis, the interface may be set to inform the user which remaining imaging features provide maximum discriminatory information to choose the most likely diagnosis. The Case Submission tools allow the user to submit a validated case and the associated imaging features to a database, which can then be used for future tuning/testing of the Bayesian networks. These submitted cases are then reviewed by an assigned expert using the provided QC tool. The Research tool presents users with cases with previously labeled features and a chosen diagnosis, for the purpose of performance evaluation. Similarly, the Education page presents cases with known features, but provides real time feedback on feature selection.

Both conventional and deep machine learning has been used to develop decision-support tools applied in medical imaging informatics. In order to take advantages of both conventional and deep learning approach, this study aims to investigate feasibility of applying a locally preserving projection (LPP) based feature regeneration algorithm to build a new machine learning classifier model to predict short-term breast cancer risk. First, a computer-aided image processing scheme was used to segment and quantify breast fibro-glandular tissue volume. Next, initially computed 44 image features related to the bilateral mammographic tissue density asymmetry were extracted. Then, an LLP-based feature combination method was applied to regenerate a new operational feature vector using a maximal variance approach. Last, a k-nearest neighborhood (KNN) algorithm based machine learning classifier using the LPP-generated new feature vectors was developed to predict breast cancer risk. A testing dataset involving negative mammograms acquired from 500 women was used. Among them, 250 were positive and 250 remained negative in the next subsequent mammography screening. Applying to this dataset, LLP-generated feature vector reduced the number of features from 44 to 4. Using a leave-onecase-out validation method, area under ROC curve produced by the KNN classifier significantly increased from 0.62 to 0.68 (p < 0.05) and odds ratio was 4.60 with a 95% confidence interval of [3.16, 6.70]. Study demonstrated that this new LPP-based feature regeneration approach enabled to produce an optimal feature vector and yield improved performance in assisting to predict risk of women having breast cancer detected in the next subsequent mammography screening.

Chest radiography (CXR) has been used as an effective tool for screening tuberculosis (TB). Because of the lack of radiological expertise in resource-constrained regions, automatic analysis of CXR is appealing as a "first reader". In addition to screening the CXR for disease, it is critical to highlight locations of the disease in abnormal CXRs. In this paper, we focus on the task of locating TB in CXRs which is more challenging due to the intrinsic difficulty of locating the abnormality. The method is based on applying a convolutional neural network (CNN) to classify the superpixels generated from the lung area. Specifically, it consists of four major components: lung ROI extraction, superpixel segmentation, multi-scale patch generation/labeling, and patch classification. The TB regions are located by identifying those superpixels whose corresponding patches are classified as abnormal by the CNN. The method is tested on a publicly available TB CXR dataset which contains 336 TB images showing various manifestations of TB. The TB regions in the images were marked by radiologists. To evaluate the method, the images are split into training, validation, and test sets with all the manifestations being represented in each set. The performance is evaluated at both the patch level and image level. The classification accuracy on the patch test set is 72.8% and the average Dice index for the test images is 0.67. The factors that may contribute to misclassification are discussed and directions for future work are addressed.

Applying deep learning technology to medical imaging informatics field has been recently attracting extensive research interest. However, the limited medical image dataset size often reduces performance and robustness of the deep learning based computer-aided detection and/or diagnosis (CAD) schemes. In attempt to address this technical challenge, this study aims to develop and evaluate a new hybrid deep learning based CAD approach to predict likelihood of a breast lesion detected on mammogram being malignant. In this approach, a deep Convolutional Neural Network (CNN) was firstly pre-trained using the ImageNet dataset and serve as a feature extractor. A pseudo-color Region of Interest (ROI) method was used to generate ROIs with RGB channels from the mammographic images as the input to the pre-trained deep network. The transferred CNN features from different layers of the CNN were then obtained and a linear support vector machine (SVM) was trained for the prediction task. By applying to a dataset involving 301 suspicious breast lesions and using a leave-one-case-out validation method, the areas under the ROC curves (AUC) = 0.762 and 0.792 using the traditional CAD scheme and the proposed deep learning based CAD scheme, respectively. An ensemble classifier that combines the classification scores generated by the two schemes yielded an improved AUC value of 0.813. The study results demonstrated feasibility and potentially improved performance of applying a new hybrid deep learning approach to develop CAD scheme using a relatively small dataset of medical images.

The UK currently has a national breast cancer-screening program and images are routinely collected from a number of screening sites, representing a wealth of invaluable data that is currently under-used. Radiologists evaluate screening images manually and recall suspicious cases for further analysis such as biopsy. Histological testing of biopsy samples confirms the malignancy of the tumour, along with other diagnostic and prognostic characteristics such as disease grade. Machine learning is becoming increasingly popular for clinical image classification problems, as it is capable of discovering patterns in data otherwise invisible. This is particularly true when applied to medical imaging features; however clinical datasets are often relatively small. A texture feature extraction toolkit has been developed to mine a wide range of features from medical images such as mammograms. This study analysed a dataset of 1,366 radiologist-marked, biopsy-proven malignant lesions obtained from the OPTIMAM Medical Image Database (OMI-DB). Exploratory data analysis methods were employed to better understand extracted features. Machine learning techniques including Classification and Regression Trees (CART), ensemble methods (e.g. random forests), and logistic regression were applied to the data to predict the disease grade of the analysed lesions. Prediction scores of up to 83% were achieved; sensitivity and specificity of the models trained have been discussed to put the results into a clinical context. The results show promise in the ability to predict prognostic indicators from the texture features extracted and thus enable prioritisation of care for patients at greatest risk.

The MACRA Act creates a Merit-Based Payment System, with monitoring patient exposure from CT providing one possible quality metric for meeting merit requirements. Quality metrics are also required by The Joint Commission, ACR, and CMS as facilities are tasked to perform reviews of CT irradiation events outside of expected ranges, review protocols for appropriateness, and validate parameters for low dose lung cancer screening. In order to efficiently collect and analyze irradiation events and associated DICOM tags, all clinical CT devices were DICOM connected to a parser which extracted dose related information for storage into a database. Dose data from every exam is compared to the appropriate external standard exam type. AAPM recommended CTDIvol values for head and torso, adult and pediatrics, coronary and perfusion exams are used for this study. CT doses outside the expected range were automatically formatted into a report for analysis and review documentation. CT Technologist textual content, the reason for proceeding with an irradiation above the recommended threshold, is captured for inclusion in the follow up reviews by physics staff. The use of a knowledge based approach in labeling individual protocol and device settings is a practical solution resulting in efficiency of analysis and review. Manual methods would require approximately 150 person-hours for our facility, exclusive of travel time and independent of device availability. An efficiency of 89% time savings occurs through use of this informatics tool including a low dose CT comparison review and low dose lung cancer screening requirements set forth by CMS.

We have developed a point-of-care imaging method for non-melanoma skin cancer surgery whereby excised tissues are imaged with a smart near infrared quenched protease probe (6qcNIR) that fluoresces in the presence of overexpressed cathepsin proteases in basal cell carcinoma (BCC) and squamous cell carcinoma (SCC), and determine if margins are clear of cancer. Here we report our imaging system and our method to validate the detection of skin cancer. We imaged skin samples with an inverted, flying spot fluorescence scanner (LI-COR Odyssey CLx). Scatter in Odyssey system was greatly reduced giving an 80% improvement in the step response as compared to a previously used macroscopic imaging system with imaging of a fluorescence phantom. We developed a validation scheme for careful comparison of fluorescent cancer signal to histology annotation, involving image segmentation, fiducial based registration and non-rigid free-form deformation, using our LI-COR fluorescence images, corresponding color images, bread-loafed tissue images, H&E slides and pathologist annotation. Spatial accuracy in the bulk of the sample was ∼500 μm. Extrapolated with a linear stretch model suggests an error at the margin of <100 μm. Cancer annotations on H&E slides were transformed and superimposed on the probe fluorescence to generate the final result. In general, the fluorescence cancer signal corresponded with histological annotation.

In digital radiography, computed radiography (CR) technology is based on latent image capture by storage phosphors whereas direct radiography (DR) technology is based either on indirect conversion using a scintillator or direct conversion using a photoconductor. DR-based portable imaging systems may enhance workflow efficiency. The purpose of this work was to investigate changes in workflow efficiency at a tertiary healthcare center after transitioning from CR to DR technology for imaging with portable x-ray units. An IRB exemption was obtained. Data for all inpatient-radiographs acquired with portable x-ray units from July-2014 till June-2015 (period 1) with CR technology (AMX4 or AMX4+ portable unit from GE Healthcare, NX workstation from Agfa Healthcare for digitization), from July-2015 till June-2016 (period 2) with DR technology (Carestream DRX-Revolution x-ray units and DRX-1C image receptors) and from July-2016 till January-2017 (period 3; same DR technology) were extracted using Centricity RIS-IC (GE Healthcare). Duration between the imaging-examination scheduled time and completed time (time_sch-com) was calculated and compared using non-parametric tests (between the three time periods with corrections for multiple comparisons; three time periods were used to identify if there were any other potential temporal trends not related to transitioning from CR to DR). IBM's SPSS package was used for statistical analysis. Overall data was obtained from 33131, 32194, and 18015 cases in periods 1, 2 and 3, respectively. Independent-Samples Kruskal-Wallis test revealed a statistically significant difference in time_sch-com across the three time periods (χ2(2, n= 83,340) = 2053, p < 0.001). The time_sch-com was highest for period 1 i.e., radiographs acquired with CR technology (median: 64 minutes) and it decreased significantly for radiographs acquired with DR technology in periods 2 (median: 49 minutes; p < 0.001) and 3 (median∶ 44 minutes; p < 0.001). Overall, adoption of DR technology resulted in a drop in time_sch-com by 27% relative to the use of CR technology. Transitioning from CR to DR was associated with improved workflow efficiency for radiographic imaging with portable x-ray units.

Retinal changes on a fundus image have been found to be related to a series of diseases. The traditional retinal image quantitative features are usually collected by various standalone and proprietary software which results in variabilities in feature extraction and data collection. Based on our previously established web-based imaging informatics platform to view DICOMized and de-identified fundus images, we developed a computer aided detection structured report (CADe SR) to capture some of the quantitative features on fundus images such as arteriole/venule diameter ratio, cup/disc diameter ratio and to record several lesions such as aneurysms, hemorrhages, neovascularization and exudates into different regions based on known research and clinically related templates such as Early Treatment Diabetic Retinopathy Study (ETDRS) 9 Region Map and four Region Map. In this way, the location patterns of the above lesions as well as morphological changes of anatomy structures could be saved in SR for further radiomics research. In addition, an on-line consultation tool was developed to facilitate further discussion among clinicians and researchers regarding any uncertainty of measurements. Compared with the present workflow of utilizing standalone software to obtain quantitative results, qualitative and quantitative data was acquired by the CADe SR directly, which will provide researchers and clinicians the ability to capture findings and will foster future image-based knowledge discovery researches.

Deep Learning (DL) has been successfully applied in numerous fields fueled by increasing computational power and access to data. However, for medical imaging tasks, limited training set size is a common challenge when applying DL. This paper explores the applicability of DL to the task of classifying a single axial slice from a CT exam into one of six anatomy regions. A total of ~29000 images selected from 223 CT exams were manually labeled for ground truth. An additional 54 exams were labeled and used as an independent test set. The network architecture developed for this application is composed of 6 convolutional layers and 2 fully connected layers with RELU non-linear activations between each layer. Max-pooling was used after every second convolutional layer, and a softmax layer was used at the end. Given this base architecture, the effect of inclusion of network architecture components such as Dropout and Batch Normalization on network performance and training is explored. The network performance as a function of training and validation set size is characterized by training each network architecture variation using 5,10,20,40,50 and 100% of the available training data. The performance comparison of the various network architectures was done for anatomy classification as well as two computer vision datasets. The anatomy classifier accuracy varied from 74.1% to 92.3% in this study depending on the training size and network layout used. Dropout layers improved the model accuracy for all training sizes.

Melanoma is the most dangerous form of skin cancer that often resembles moles. Dermatologists often recommend regular skin examination to identify and eliminate Melanoma in its early stages. To facilitate this process, we propose a hand-held computer (smart-phone, Raspberry Pi) based assistant that classifies with the dermatologist-level accuracy skin lesion images into malignant and benign and works in a standalone mobile device without requiring network connectivity. In this paper, we propose and implement a hybrid approach based on advanced deep learning model and domain-specific knowledge and features that dermatologists use for the inspection purpose to improve the accuracy of classification between benign and malignant skin lesions. Here, domain-specific features include the texture of the lesion boundary, the symmetry of the mole, and the boundary characteristics of the region of interest. We also obtain standard deep features from a pre-trained network optimized for mobile devices called Google's MobileNet. The experiments conducted on ISIC 2017 skin cancer classification challenge demonstrate the effectiveness and complementary nature of these hybrid features over the standard deep features. We performed experiments with the training, testing and validation data splits provided in the competition. Our method achieved area of 0.805 under the receiver operating characteristic curve. Our ultimate goal is to extend the trained model in a commercial hand-held mobile and sensor device such as Raspberry Pi and democratize the access to preventive health care.

The increasing incidence of diabetes mellitus (DM) in modern society has become a serious issue. DM can also lead to several secondary clinical complications. One of these complications is diabetic retinopathy (DR), which is the leading cause of new cases of blindness for adults in the United States. While DR can be treated if screened and caught early in progression, the only currently effective method to detect symptoms of DR in the eyes of DM patients is through the manual analysis of fundus images. Manual analysis of fundus images is time-consuming for ophthalmologists and can reduce access to DR screening in rural areas. Therefore, effective automatic prescreening tools on a cloud-based platform might be a potential solution to that problem. Recently, deep learning (DL) approaches have been shown to have state-of-the-art performance in image analysis tasks. In this study, we established a research PACS for fundus images to view DICOMized and anonymized fundus images. We prototyped a deep learning engine in the PACS server to perform prescreening classification of uploaded fundus images into DR grade. We fine-tuned a deep convolutional neural network (CNN) model pretrained on the ImageNet dataset by using over 30,000 labeled image samples from the public Kaggle Diabetic Retinopathy Detection fundus image dataset⁶. We linked the PACS repository with the DL engine and demonstrated the output predicted result of DR into the PACS worklist. The initial prescreened result was promising and such applications could have potential as a “second reader” with future CAD development for nextgeneration PACS.

The number of health-care associated infections is increasing worldwide. Hand hygiene has been identified as one of the most crucial measures to prevent bacteria from spreading. However, compliance with recommended procedures for hand hygiene is generally poor, even in modern, industrialized regions. We present an optical assistance system for monitoring the hygienic hand disinfection procedure which is based on machine learning. Firstly, each hand and underarm of a person is detected in a down-sampled 96 px x 96 px depth video stream by pixelwise classification using a fully convolutional network. To gather the required amount of training data, we present a novel approach in automatically labeling recorded data using colored gloves and a color video stream that is registered to the depth stream. The colored gloves are used to segment the depth data in the training phase. During inference, the colored gloves are not required. The system detects and separates detailed hand parts of interacting, self-occluded hands within the observation zone of the sensor. Based on the location of the segmented hands, a full resolution region of interest (ROI) is cropped. A second deep neural network classifies the ROI into ten separate process steps (gestures), with nine of them based on the recommended hand disinfection procedure of the World Health Organization, and an additional error class. The combined system is cross-validated with 21 subjects and predicts with an accuracy of 93.37% (± 2.67%) which gesture is currently executed. The feedback is provided with 30 frames per second.

Deep convolutional neural networks have found success in semantic image segmentation tasks in computer vision and medical imaging. These algorithms are executed on conventional von Neumann processor architectures or GPUs. This is suboptimal. Neuromorphic processors that replicate the structure of the brain are better-suited to train and execute deep learning models for image segmentation by relying on massively-parallel processing. However, given that they closely emulate the human brain, on-chip hardware and digital memory limitations also constrain them. Adapting deep learning models to execute image segmentation tasks on such chips, requires specialized training and validation.

In this work, we demonstrate for the first-time, spinal image segmentation performed using a deep learning network implemented on neuromorphic hardware of the IBM TrueNorth Neurosynaptic System and validate the performance of our network by comparing it to human-generated segmentations of spinal vertebrae and disks. To achieve this on neuromorphic hardware, the training model constrains the coefficients of individual neurons to {-1,0,1} using the Energy Efficient Deep Neuromorphic (EEDN)1 networks training algorithm. Given the ∼1 million neurons and 256 million synapses, the scale and size of the neural network implemented by the IBM TrueNorth allows us to execute the requisite mapping between segmented images and non-uniform intensity MR images >20 times faster than on a GPU-accelerated network and using <0.1 W. This speed and efficiency implies that a trained neuromorphic chip can be deployed in intra-operative environments where real-time medical image segmentation is necessary.

The benign and malignant differential diagnosis of small pulmonary nodules (diameter < 20 mm) found in lung CT images is big challenges for most of radiologists. Here, we presented our preliminary study of benign and malignant differentiation of small pulmonary nodules in lung CT images by using deep learning Convolutional Neural Network (CNN). The 921 cases with small benign and malignant pulmonary nodules confirmed by pathology were collected from three data sources and were used to train and validate the CNN. The preliminary results of AUCs of ROC curves for differentiating benign and malignant pulmonary small nodules with various types and sizes of solid, semi-solid and ground glass nodules were presented and discussed.

Chronic wounds affect millions of people around the world. In particular, elderly persons in home care may develop decubitus. Here, mobile image acquisition and analysis can provide a good assistance. We develop a system for mobile wound capture using mobile devices such as smartphones. The photographs are acquired with the integrated camera of the device and then calibrated and processed to determine the size of various tissues that are present in a wound, i.e., necrotic, sloughy, and granular tissue. The random forest classifier based on various color and texture features is used for that. These features are Sobel, Hessian, membrane projections, variance, mean, median, anisotropic diffusion, and bilateral as well as Kuwahara filters. The resultant probability output is thresholded using the Otsu technique. The similarity between manual ground truth labeling and the classification is measured. The acquired results are compared to those achieved with a basic technique of color thresholding, as well as those produced by the SVM classifier. The fast random forest was found to produce better results. It is also seen to have a superior performance when the method is applied only to the wound regions having the background subtracted. Mean similarity is 0.89, 0.39, and 0.44 for necrotic, sloughy, and granular tissue, respectively. Although the training phase is time consuming, the trained classifier performs fast enough to be implemented on the mobile device. This will allow comprehensive monitoring of skin lesions and wounds.

The goal of this study was to investigate the survival prediction of squamous cell head and neck cancer (SCHNC) patients by using radiomic features that were selected using an artificial neural network (ANN). We employed computed tomography (CT) images of 86 squamous cell lung cancer (SCLC) patients for the feature selection, and 30 SCHNC patients for a test of the selected features. 486 radiomic features, i.e., statistic, texture, wavelet-based features, were extracted from the tumor regions in the CT images. The ANN was constructed for selecting 10 features that could classify the SCLC patients into shorter and longer survival groups than 2 years. The features were selected based on weights with strong links between the features and predicted survival in ANN. The survival times of the SCHNC patients, who were divided into two groups with respect to the median of each of the top 10 ranked features, were estimated using a Kaplan-Meier method. The statistical significant differences between survival curves of the two groups were assessed for the 10 features using a log-rank test. The homogeneity feature of the wavelet-based HHL image (HHL_Homogeneity) demonstrated a statistically significant difference (p < 0.01) between the two groups of SCHNC, but the other 9 features did not. Our results suggest that the 2-year survival of the SCHNC patients could be predicted by using at least the radiomic feature selected among the features for SCLC patients using the ANN-based feature selection approach.

With the development of wireless capsule endoscopy (WCE) and its extensive applications in clinic, doctors need to spend more time on reviewing the WCE images for lesions diagnosis. Therefore, automatic lesion detection for WCE has gradually become a research hotspot, which aims to reduce the pressure of doctors and improves the diagnosis efficiency. Many researchers adopted the traditional machine learning method to realize polyp detections, however, these methods need to extract the features manually, which were unable to find higher features of WCE images. So, in this study, we proposed a novel method that based on convolution neural network (CNN) to automatically recognize polyp in small bowel WCE image. We utilized the Alexnet architecture, one of the classical CNN, to extract the features of WCE images and classify polyp images from normal ones. We selected 14408 images from different patients, including 408 polyp images and 14000 normal images. Since the amount of initial polyp images is small, then, we did the data augmentation, including rotation, luminance change, blurring, and noise. At last the experimental results demonstrated that the method we proposed had a promising performance in polyp detection, whose accuracy, sensitivity and specificity can reach at 99.88%, 99.40% and 99.93%, respectively. Additionally, we evaluated ROC curve and its AUC value, which further confirmed that our model has a high accuracy and reliability in polyp detection. This proposed method has great potential to be used in the clinical examination to help doctors from the tedious image reviewing work.

Background. Prostate segmentation is a crucial step in computer-aided systems for prostate cancer detection. Multi-planar acquisitions are commonly used by clinicians to obtain a more accurate patient diagnosis but their relevance in prostate segmentation using fully automated algorithms has not been assessed. To date, the limited assessment of this relevance stems from the fact that both axial and sagittal prostate imaging views, as opposed to a single view, doubles the acquisition time. In this work, we assess the relevance of multi-planar imaging for prostate segmentation within a deep learning segmentation framework. Materials and Methods. We propose a deep learning prostate segmentation framework either from either axial or from axial and sagittal T2-weighted magnetic resonance images (MRI). The system is based on an ensemble of convolutional neural networks, each independently trained on a single imaging view. We compare single-view (axial) segmentations to those obtained from two imaging views (axial and sagittal) to assess the relevance of using multi-planar acquisitions. Algorithm performance assessment will be two-fold: 1) the global DICE score between the algorithm’s predictions and the segmentations of an experienced reader will be computed and 2) the number of lesions located within the algorithm’s segmentation prediction will be calculated. A subset of 80 patients from the public PROSTATEx-2 database containing both axial and sagittal T2-weighted MRIs will be used for this study. Results. The multiplanar network outperformed the network trained on only axial views according to both the proposed metrics. A statistically significant increase of 4% in DICE scores was found along with an 9% increase in the number of lesions within the predicted segmentation. Conclusions. The proposed method allows for a fully automatic segmentation of the prostate from single- or multi-view MRI and assesses the relevance of multi-planar MRI acquisitions for fully automatic prostate segmentation algorithms.

In human brain, Corpus Callosum (CC) is the largest white matter structure, connecting between right and left hemispheres. Structural features such as shape and size of CC in midsagittal plane are of great significance for analyzing various neurological diseases, for example Alzheimer’s disease, autism and epilepsy. For quantitative and qualitative studies of CC in brain MR images, robust segmentation of CC is important. In this paper, we present a novel method for CC segmentation. Our approach is based on deep neural networks and the prior information generated from multi-atlas images. Deep neural networks have recently shown good performance in various image processing field. Convolutional neural networks (CNN) have shown outstanding performance for classification and segmentation in medical image fields. We used convolutional neural networks for CC segmentation. Multi-atlas based segmentation model have been widely used in medical image segmentation because atlas has powerful information about the target structure we want to segment, consisting of MR images and corresponding manual segmentation of the target structure. We combined the prior information, such as location and intensity distribution of target structure (i.e. CC), made from multi-atlas images in CNN training process for more improving training. The CNN with prior information showed better segmentation performance than without.

Accurate segmentation and measurement of brain tumors plays an important role in clinical practice and research, as it is critical for treatment planning and monitoring of tumor growth. However, brain tumor segmentation is one of the most challenging tasks in medical image analysis. Since manual segmentations are subjective, time consuming and neither accurate nor reliable, there exists a need for objective, robust and fast automated segmentation methods that provide competitive performance. Therefore, deep learning based approaches are gaining interest in the field of medical image segmentation. When the training data set is large enough, deep learning approaches can be extremely effective, but in domains like medicine, only limited data is available in the majority of cases. Due to this reason, we propose a method that allows to create a large dataset of brain MRI (Magnetic Resonance Imaging) images containing synthetic brain tumors - glioblastomas more specifically - and the corresponding ground truth, that can be subsequently used to train deep neural networks.

We present crowdsourcing as an additional modality to aid radiologists in the diagnosis of lung cancer from clinical chest computed tomography (CT) scans. More specifically, a complete work flow is introduced which can help maximize the sensitivity of lung nodule detection by utilizing the collective intelligence of the crowd. We combine the concept of overlapping thin-slab maximum intensity projections (TS-MIPs) and cine viewing to render short videos that can be outsourced as an annotation task to the crowd. These videos are generated by linearly interpolating overlapping TS-MIPs of CT slices through the depth of each quadrant of a patient's lung. The resultant videos are outsourced to an online community of non-expert users who, after a brief tutorial, annotate suspected nodules in these video segments. Using our crowdsourcing work flow, we achieved a lung nodule detection sensitivity of over 90% for 20 patient CT datasets (containing 178 lung nodules with sizes between 1-30mm), and only 47 false positives from a total of 1021 annotations on nodules of all sizes (96% sensitivity for nodules>4mm). These results show that crowdsourcing can be a robust and scalable modality to aid radiologists in screening for lung cancer, directly or in combination with computer-aided detection (CAD) algorithms. For CAD algorithms, the presented work flow can provide highly accurate training data to overcome the high false-positive rate (per scan) problem. We also provide, for the first time, analysis on nodule size and position which can help improve CAD algorithms.

The demand for medical images for research is ever increasing owing to the rapid rise in novel machine learning approaches for early detection and diagnosis. The OPTIMAM Medical Image Database (OMI-DB)1,2 was created to provide a centralized, fully annotated dataset for research. The database contains both processed and unprocessed images, associated data, annotations and expert-determined ground truths. Since the inception of the database in early 2011, the volume of images and associated data collected has dramatically increased owing to automation of the collection pipeline and inclusion of new sites. Currently, these data are stored at each respective collection site and synced periodically to a central store. This leads to a large data footprint at each site, requiring large physical onsite storage, which is expensive.

Here, we propose an update to the OMI-DB collection system, whereby the storage of all the data is automatically transferred to the cloud on collection. This change in the data collection paradigm reduces the reliance of physical servers at each site; allows greater scope for future expansion; and removes the need for dedicated backups and improves security. Moreover, with the number of applications to access the data increasing rapidly with the maturity of the dataset cloud technology facilities faster sharing of data and better auditing of data access. Such updates, although may sound trivial; require substantial modification to the existing pipeline to ensure data integrity and security compliance. Here, we describe the extensions to the OMI-DB collection pipeline and discuss the relative merits of the new system.

We have innovatively introduced Visual Patient (VP) concept and method visually to represent and index patient imaging diagnostic records (IDR) in last year SPIE Medical Imaging (SPIE MI 2017), which can enable a doctor to review a large amount of IDR of a patient in a limited appointed time slot. In this presentation, we presented a new approach to design data processing architecture of VP system (VPS) to acquire, process and store various kinds of IDR to build VP instance for each patient in hospital environment based on Hadoop distributed processing structure. We designed this system architecture called Medical Information Processing System (MIPS) with a combination of Hadoop batch processing architecture and Storm stream processing architecture. The MIPS implemented parallel processing of various kinds of clinical data with high efficiency, which come from disparate hospital information system such as PACS, RIS LIS and HIS.

Online peer to peer medical consultation between doctors such as physicians and specialists in China has a broad market demand and has been continuously accepted. For some difficult diseases, electronic medical records with medical images are required to present to both sides at same time during the consultation so that both sides can manipulate the records interactively to understand the medical meanings of the records, especially images. Here, we presented design of a teleconsultation system integrated with a cloud-based collaborative image sharing network to provide online peer-to-peer medical consultation for difficult cases with multi-media medical records including DICOM images. The presented teleconsultation system provides bidirectional interactive manipulations on images presented to peer-to-peer sides and has been used for small lung nodule diagnosis services between Huadong hospital in Shanghai and Jiaxing First Hospital in Zhejiang Province through Internet.

This work presents how Virtual Reality (VR) can easily be integrated into medical applications via a plugin for a medical image processing framework called MeVisLab. A multi-threaded plugin has been developed using OpenVR, a VR library that can be used for developing vendor and platform independent VR applications. The plugin is tested using the HTC Vive, a head-mounted display developed by HTC and Valve Corporation.

Authentication and copyright identification are two critical security issues for medical images. Although zerowatermarking schemes can provide durable, reliable and distortion-free protection for medical images, the existing zerowatermarking schemes for medical images still face two problems. On one hand, they rarely considered the distinguishability for medical images, which is critical because different medical images are sometimes similar to each other. On the other hand, their robustness against geometric attacks, such as cropping, rotation and flipping, is insufficient. In this study, a novel discriminative and robust zero-watermarking (DRZW) is proposed to address these two problems. In DRZW, content-based features of medical images are first extracted based on completed local binary pattern (CLBP) operator to ensure the distinguishability and robustness, especially against geometric attacks. Then, master shares and ownership shares are generated from the content-based features and watermark according to (2,2) visual cryptography. Finally, the ownership shares are stored for authentication and copyright identification. For queried medical images, their content-based features are extracted and master shares are generated. Their watermarks for authentication and copyright identification are recovered by stacking the generated master shares and stored ownership shares. 200 different medical images of 5 types are collected as the testing data and our experimental results demonstrate that DRZW ensures both the accuracy and reliability of authentication and copyright identification. When fixing the false positive rate to 1.00%, the average value of false negative rates by using DRZW is only 1.75% under 20 common attacks with different parameters.

This proposed method aims towards a full automation of the detection of coronary artery blockage through some image processing techniques so that the system does not have to rely on human's inspection. The goal of the research is to implement the proposed image processing techniques so the system can detect the narrowing area of the wall of coronary arteries due to the condensation of different artery blocking agents. The research suggests that the system will require a 64-slice CTA image as input. After the acquisition of the desired input image, it will go through several steps to determine the region of interest. This research proposes a two stage approach that includes the preprocessing stage and decision stage. The pre-processing stage involves common image processing strategies while the decision stage involves the extraction and calculation of two feature ratios to finally determine the intended result. In order to get more insights of the subject of these examinations, this research has proposed the use of an algorithm to create a 3-D model.

Segmentation of brain images especially into three main tissue types: Gray Matter (GM), Cerebrospinal Fluid (CSF), and White Matter (WM) has important role in computer aided neurosurgery and diagnosis. In imaging, physical phenomena and the acquisition system are responsible for noise and the Partial Volume Effect (PVE) respectively, which affect the uncertainty and the imprecision. To reduce the effect of these different imperfections, we propose a clustering approach that is based on a fuzzy- possibilistic segmentation process for the assessment of WM, GM and CSF volumes from Alzheimer’s brain images. The brain segmentation scheme which is illustrated in the study of Alzheimer’s disease using Alzheimer’s disease Neuroimaging Initiative (ADNI) and real images take in consideration the PVE and it is less sensitive to noise.

OIPAV (Ophthalmic Images Processing, Analysis and Visualization) is a cross-platform software which is specially oriented to ophthalmic images. It provides a wide range of functionalities including data I/O, image processing, interaction, ophthalmic diseases detection, data analysis and visualization to help researchers and clinicians deal with various ophthalmic images such as optical coherence tomography (OCT) images and color photo of fundus, etc. It enables users to easily access to different ophthalmic image data manufactured from different imaging devices, facilitate workflows of processing ophthalmic images and improve quantitative evaluations. In this paper, we will present the system design and functional modules of the platform and demonstrate various applications. With a satisfying function scalability and expandability, we believe that the software can be widely applied in ophthalmology field.

In addition to length and weight, maximum head circumference is a standard measurement obtained when evaluating pediatric patients, especially in the first two years of life. This study describes the technique to accurately and anatomically determine the head circumference from MRI studies within the picture archiving and communications system and adds the numerical value to the final radiology report as a standard application. Correlation between the automatic and manual measurements was within 2% of error. An accurate automatic determination of head's circumference from MRI images is of definite clinical usefulness.

We investigated the effect of radiomic texture-curvature (RTC) features of lung CT images in the prediction of the overall survival of patients with rheumatoid arthritis-associated interstitial lung disease (RA-ILD). We retrospectively collected 70 RA-ILD patients who underwent thin-section lung CT and serial pulmonary function tests. After the extraction of the lung region, we computed hyper-curvature features that included the principal curvatures, curvedness, bright/dark sheets, cylinders, blobs, and curvature scales for the bronchi and the aerated lungs. We also computed gray-level co-occurrence matrix (GLCM) texture features on the segmented lungs. An elastic-net penalty method was used to select and combine these features with a Cox proportional hazards model for predicting the survival of the patient. Evaluation was performed by use of concordance index (C-index) as a measure of prediction performance. The C-index values of the texture features, hyper-curvature features, and the combination thereof (RTC features) in predicting patient survival was estimated by use of bootstrapping with 2,000 replications, and they were compared with an established clinical prognostic biomarker known as the gender, age, and physiology (GAP) index by means of two-sided t-test. Bootstrap evaluation yielded the following C-index values for the clinical and radiomic features: (a) GAP index: 78.3%; (b) GLCM texture features: 79.6%; (c) hypercurvature features: 80.8%; and (d) RTC features: 86.8%. The RTC features significantly outperformed any of the other predictors (P < 0.001). The Kaplan-Meier survival curves of patients stratified to low- and high-risk groups based on the RTC features showed statistically significant (P < 0.0001) difference. Thus, the RTC features can provide an effective imaging biomarker for predicting the overall survival of patients with RA-ILD.

Relying solely on ophthalmic equipment is unable to meet the present health needs. It is urgent to find an efficient way to provide a quick screening and early diagnosis on diabetic retinopathy and other ophthalmic diseases. The purpose of this study is to develop a cloud-base system for medical image especially ophthalmic image to store, view and process and accelerate the screening and diagnosis. In this purpose the system with web application, upload client, storage dependency and algorithm support is implemented. After five alpha tests, the system bore the thousands of large traffic access and generated hundreds of reports with diagnosis.

In this paper, we present a thermal imaging based app to augment traditional appearance based wound growth monitoring. Accurate diagnose and track of wound healing enables physicians to effectively assess, document, and individualize the treatment plan given to each wound patient. Currently, wounds are primarily examined by physicians through visual appearance and wound area. However, visual information alone cannot present a complete picture on a wound’s condition. In this paper, we use a smartphone attached thermal imager and evaluate its effectiveness on augmenting visual appearance based wound diagnosis. Instead of only monitoring wound temperature changes on a wound, our app presents physicians a comprehensive measurements including relative temperature, wound healing thermal index, and wound blood flow. Through the rat wound experiments and by monitoring the integrated thermal measurements over 3 weeks of time frame, our app is able to show the underlying healing process through the blood flow. The implied significance of our app design and experiment includes: (a) It is possible to use a low cost smartphone attached thermal imager for added value on wound assessment, tracking, and treatment; and (b) Thermal mobile app can be used for remote wound healing assessment for mobile health based solution.

The main goal of sports science and performance enhancement is to collect video and image data, process them, and quantify the results, giving insight to help athletes improve technique. For long jump in track and field, the processed output of video with force vector overlays and force calculations allow coaches to view specific stages of the hop, step, and jump, and identify how each stage can be improved to increase jump distance. Outputs also provide insight into how athletes can better maneuver to prevent injury. Currently, each data collection site collects and stores data with their own methods. There is no standard for data collection, formats, or storage. Video files and quantified results are stored in different formats, structures, and locations such as Dropbox and hard drives. Using imaging informatics-based principles we can develop a platform for multiple institutions that promotes the standardization of sports performance data. In addition, the system will provide user authentication and privacy as in clinical trials, with specific user access rights. Long jump data collected from different field sites will be standardized into specified formats before database storage. Quantified results from image-processing algorithms are stored similar to CAD algorithm results. The system will streamline the current sports performance data workflow and provide a user interface for athletes and coaches to view results of individual collections and also longitudinally across different collections. This streamlined platform and interface is a tool for coaches and athletes to easily access and review data to improve sports performance and prevent injury.

For human performance analysis, extensive experimental trials are often conducted to identify the underlying cause or long-term consequences of certain pathologies and to improve motor functions by examining the movement patterns of affected individuals. Data collected for human performance analysis includes high-speed video, surveys, spreadsheets, force data recordings from instrumented surfaces etc. These datasets are recorded from various standalone sources and therefore captured in different folder structures as well as in varying formats depending on the hardware configurations. Therefore, data integration and synchronization present a huge challenge while handling these multimedia datasets specifically for large datasets. Another challenge faced by researchers is querying large quantity of unstructured data and to design feedbacks/reporting tools for users who need to use datasets at various levels. In the past, database server storage solutions have been introduced to securely store these datasets. However, to automate the process of uploading raw files, various file manipulation steps are required. In the current workflow, this file manipulation and structuring is done manually and is not feasible for large amounts of data. However, by attaching metadata files and data dictionaries with these raw datasets, they can provide information and structure needed for automated server upload. We introduce one such system for metadata creation for unstructured multimedia data based on the DICOM data model design. We will discuss design and implementation of this system and evaluate this system with data set collected for movement analysis study. The broader aim of this paper is to present a solutions space achievable based on medical imaging informatics design and methods for improvement in workflow for human performance analysis in a biomechanics research lab.