A Novel Approach to Detecting Melanoma Using Deep Learning

Derek Sheen

he/him | age 15 | Waterloo, ON

Waterloo-Wellington Science Fair Gold Medallion, Best Engineering Division Award, and Award of Merit | Canada-Wide Science Fair Bronze Medallion and Actuarial Foundation of Canada Award

VIEW PDF

INTRODUCTION

Melanoma Cancer
Melanoma is the deadliest type of skin cancer and is the fifth most common cancer in the United States (Patel et al., 2022). If untreated, melanoma can spread to other parts of the body, such as organs or lymph nodes, drastically decreasing the survival rate (MacGill, 2019). Melanoma can be caused by many different internal factors such as age, number of moles, and genetics (MacGill, 2019). However, avoiding overexposure to UV radiation is one of the few risk factors that can be controlled, significantly lowering the risk of skin cancer (MacGill, 2019). The detection of melanoma in-situ has a survival rate of 99%, while stage IV melanoma has a survival rate of only 18% (Venosa & Kleiman, 2021). Therefore, the earlier one detects the symptoms of melanoma, the higher chances of a patient being cured (Kadampur, 2019). However, in 2020, there were 57,043 deaths worldwide, largely because of the lack of early detection (Patel et al., 2022).

Methods to Diagnose Melanoma
Dermoscopy is used for the screening of melanoma, which uses a specific criterion such as the ABCD rule or the Menzies method for analyzing irregular dots, streaks, and blotches (Feng, 2019). Next, biopsies are used to take samples of the suspected skin for examination in a laboratory (MacGill, 2019). However, melanoma is often left untreated. People avoid seeking treatment as biopsies are expensive, too far away, too inconvenient, and require a referral from a family doctor. At the Skin Cancer Screening Clinic (2022), biopsies can cost $55 - $175, with regular checkups leading to early diagnosis. Also, biopsies can take 1-2 weeks to get results (Gardner, 2020). This can be critical to one’s survival rate, as it can take only 6 weeks for melanoma to become life-threatening (Better Health Channel, 2022).

Rationale and Hypothesis
Developing a deep learning algorithm to determine whether the lesion is melanoma or not, would provide an alternative to biopsies that has no costs with instant results. In addition, it may be used by clinics for a more thorough diagnosis of melanoma. There has been much research done on melanoma and deep learning. The Khulna University of Engineering & Technology (Rasul et al., 2020) utilized deep learning architectures such as ResNet500, Xception, VGG16, and InceptionV3 with accuracies of 95%, 96%, 94%, and 95% respectively. My supervised deep learning algorithm utilizes MobileNet V2, which through many processes, would be able to predict melanoma at 99% accuracy. Furthermore, the goal is to make a lightweight web application accessible on all devices where users would be able to upload images of their moles into the deep learning model to accurately provide a diagnosis in the comfort of their homes.

METHOD & MATERIALS

Dataset
The HAM10K dataset from Kaggle was used to train my deep learning model, which is a large collection of multi-source dermatoscopic images of common pigmented skin lesions (Kaggle, 2018). By using various images in my sample dataset, my deep learning algorithm would be able to find patterns in melanoma-positive skin to predict melanoma in new images and datasets. The dataset consisted of 8,903 melanoma images and 8,902 non-melanoma images. The dataset was partitioned into three separate folders for training, testing, and validation. 60% of this data will be used in the training process for the deep learning model.

Preprocessing
Images were zoomed in, rotated, and reflected for data augmentation. Other data augmentation techniques that I used were vertical shifting, horizontal shifting, and shearing. The deep learning algorithm will need to adjust to different picture sizes, orientations, and qualities, as it will need to crop or resize the image and address all these possible impacts so that it can be processed correctly.

MobileNet V2 Architecture
After pre-processing, images were resized to 224 by 224 by 3 for the implementation of MobileNet V2 and TensorFlow 2 SavedModel. MobileNet V2 allows for my deep learning model to be lightweight, almost twice as fast as MobileNet V1 (Hollemans, 2019). Due to RGB pixel values ranging from zero to two hundred fifty-six, they were divided by two-hundred fifty-five for normalization so that the color values ranged from 0-1, allowing for faster computation (Ponraj, 2021). Then, the convolutional neural networks were customized into batch sizes of 16 with randomized pre-processed images.

CNN Model Architecture
The deep learning model architecture consisted of the Keras Layer, exported from MobileNet V2, in addition to a series of hidden and output layers shown in Figure 1. The MobileNet V2 architecture consisted of inverted bottleneck residual blocks and lightweight convolutional layers. The residual bottleneck consisted of a point-wise convolution to expand features, depth-wise separable convolution, and point-wise convolution to reduce features (Tajbakhsh, 2020). MobileNet V2 also uses linear inverted bottlenecks that use linear activation functions for memory efficiency (Tajbakhsh, 2020). After, data is passed into a flatten layer, which is then inputted into a reLU dense layer with 512 neurons for a lower training loss. A Dropout Layer was used at 20% to reduce overfitting. Finally, the melanoma prediction was made using a softmax layer for binary classification. A confidence interval was also calculated using Numpy to distinguish between strong and weak melanoma predictions.

**Figure 1:** Deep Learning Model Architecture (Sandler et al., 2018). The dermoscopic image is passed into the MobileNet V2 Architecture as the input. After, the Flatten Layer converts the output from MobileNet V2 into a 1-dimensional array to be inputted into the Dense Layer, which classifies the image through Softmax and outputs 1 or 0, representing non-melanoma and melanoma respectively.

Tests and Experiments
The batch size, steps per epoch, and epochs were tested to find the best accuracy. The initial training model had a batch size of 16, 10 steps per epoch, and 15 epochs. Through changing the steps per epoch from 10 to 455, which is the train dataset length divided by the batch size, there was an increase of 0.5% in accuracy. By changing the epoch value from 15 to 25, there was no change in accuracy. After training, Scikit-learn and MatPlotLib were used to create a confusion matrix using the validation dataset as seen in Figure 2.

**Figure 2**: The Confusion Matrix represents the correctly and incorrectly predicted melanoma and non-melanoma images.

**Figure 3**: Streamlit Web Application predicts a melanoma dermoscopy correctly at an 87% confidence level.

In Figure 3, Streamlit and Flask-Ngrok were used to create a web application for users to access the deep model. Input images were pre-processed and downloaded into Google Cloud. The image was converted into an array using OpenCV and then passed into the deep learning model to provide a prediction and confidence rate. All modeling was performed using TensorFlow and Keras on a Windows Intel Core i3 processor.

OBSERVATIONS & RESULTS

The deep learning model had an accuracy of 99.04%, with 455 steps per epoch, a batch size of 16, and 15 epochs. From analyzing the evaluation metrics in Figure 4, there was 98% precision for melanoma, 100% precision for non-melanoma, 100% recall for melanoma, 98% recall for non-melanoma, and 99% for both f1-scores. The evaluation metrics, such as the f1-score which provides the mean of how precise and accurate the results were, are essential to validate the accuracy of the deep learning model, The number of epochs remained the same throughout these tests, as I decided that fifteen epochs were a reasonable amount.

**Figure 4**: Classification Report of Deep Learning Model. The precision represents the number of correctly classified melanoma images from the total number of correctly and incorrectly predicted melanoma classified images. The recall represents the number of correctly classified melanoma images from the correctly predicted melanoma images and incorrectly predicted non-melanoma classified images. The F1 score is the mean of the precision and recall while support represents the number of melanoma occurrences in the dataset (Kharwal, 2021).

DISCUSSION

The process of image processing through Keras’ neural networks is efficient as MobileNet’s runtime and memory usage is adequate and lightweight for optimization of phones and computers in a practical environment, such as labs and hospitals. A prediction is generated through convolutional neural networks created using Keras Layers. It is accessible through a web app, allowing for ease of access while displaying a confidence level. This application can have large global impacts in both developed and developing countries, due to its accessibility and ease of use. It offers a cheap alternative to biopsies and enables the development of open access melanoma scans on a global scale. While it is useful for everyday people, it also provides practical usage at hospitals for an earlier and more accurate diagnosis of melanoma, advancing the fields of dermatology and anatomical pathology.

CONCLUSION

Overall, this study aimed to detect and diagnose melanoma that is accessible through a web application. It was concluded that the deep learning model could achieve this by developing a convolutional neural network using TensorFlow and Keras at a 99% accuracy rate.

ACKNOWLEDGEMENTS

I would like to thank my family, friends, and the WWSEF team for encouraging me to pursue this project.

REFERENCES

Better Health Channel. (n.d.). Better Health Channel. Retrieved March 23, 2022, from https://www.betterhealth.vic.gov.au/health/conditionsandtreatments/melanoma

Feng, Y., Feng, J., & Bao, J. (2019, August 21). Dermoscopy of Melanoma and Non-melanoma Skin Cancers. Frontiers. Retrieved April 27, 2022, from https://www.frontiersin.org/articles/10.3389/fmed.2019.00180/full

Gardner, S. S. (2020, October 15). Skin Biopsy: Purpose, Procedure, Complications, Recovery. WebMD. Retrieved April 27, 2022, from https://www.webmd.com/melanoma-skin-cancer/skin-biopsies

Hollemans, M. (2018, April 22). MobileNet version 2. Machine, Think! Retrieved April 28, 2022, from https://machinethink.net/blog/mobilenet-v2/

Kadampur, M. A. (2019). Skin cancer detection: Applying a deep learning based model driven architecture in the cloud for classifying dermal cell images. Retrieved March 23, 2022, from https://www.sciencedirect.com/science/article/pii/S2352914819302047

Kharwal, A. (2021, July 7). Classification Report in Machine Learning. Thecleverprogrammer. Retrieved September 7, 2022, from https://thecleverprogrammer.com/2021/07/07/classification-report-in-machine-learning/

MacGill, M. (2019, November 7). Melanoma: Stages, types, causes, and pictures. Medical News Today. Retrieved April 27, 2022, from https://www.medicalnewstoday.com/articles/154322#risk-factors

Patel, J. D. et al. (2022, February). Melanoma: Statistics. Cancer.Net. Retrieved April 26, 2022, from https://www.cancer.net/cancer-types/melanoma/statistics

Ponraj, A. (2021, February 20). A Tip A Day – Python Tip #8: Why should we Normalize image pixel values or divide by 255? » DevSkrol. DevSkrol. Retrieved April 21, 2022, from https://medium.com/analytics-vidhya/a-tip-a-day-python-tip-8-why-should-we-normalize-image-pixel-values-or-divide-by-255-4608ac5cd26a

Rasul, F., Kumar Dey, N., & Hashem, M.M.A. (2020, June 7). ResearchGate. A Comparative Study of Neural Network Architectures for Lesion Segmentation and Melanoma Detection. Retrieved April 27, 2022, from https://www.researchgate.net/publication/342549085_A_Comparative_Study_of_Neural_Network_Architectures_for_Lesion_Segmentation_and_Melanoma_Detection

Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018, January 13). MobileNetV2: Inverted Residuals and Linear Bottlenecks. MobileNetV2: Inverted Residuals and Linear Bottlenecks. Retrieved April 28, 2022, from https://arxiv.org/abs/1801.04381

Scarlat, A. (2019). melanoma. Kaggle. Retrieved April 27, 2022, from https://www.kaggle.com/drscarlat/melanoma

Skin Cancer Screening Clinic. (n.d.). Skin Cancer Screening Clinic. Early Skin Cancer Detection Clinic. Retrieved April 27, 2022, from https://www.skincancerscreening.ca/

Tajbakhsh, N. (2020, April 4). Explaining memory-efficient inference of MobileNet v2.0 | by Nima Tajbakhsh. Medium. Retrieved April 28, 2022, from https://medium.com/@nimatajbakhsh/explaining-memory-efficient-inference-of-mobilenet-v2-0-10848e2bc622

Venosa, A., & Kleiman, J. (2021, October 10). How Dangerous is Melanoma? It's All a Matter of Timing. Skin Cancer Foundation. Retrieved April 26, 2022, from https://www.skincancer.org/blog/dangerous-melanoma-matter-timing/

von Wangenheim, A. (2019, July 12). Deep Learning for Diagnosis of Skin Images with fastai. Towards Data Science. Retrieved April 27, 2022, from https://towardsdatascience.com/deep-learning-for-diagnosis-of-skin-images-with-fastai-792160ab5495

BIBLIOGRAPHY

Classify Images Using Python & Machine Learning. (2020, June 5). YouTube. Retrieved March 23, 2022, from https://www.youtube.com/watch?v=iGWbqhdjf2s

Image Classification using CNN | Deep Learning Tutorial | Machine Learning Project 9 | Edureka. (2021, August 6). YouTube. Retrieved March 23, 2022, from https://www.youtube.com/watch?v=ejkRh9obVjk

Image Preparation for Convolutional Neural Networks with TensorFlow's Keras API. (2020, July 16). YouTube. Retrieved March 23, 2022, from https://www.youtube.com/watch?v=_L2uYfVV48I

Keras with TensorFlow Course - Python Deep Learning and Neural Networks for Beginners Tutorial. (2020, June 18). YouTube. Retrieved March 23, 2022, from https://www.youtube.com/watch?v=qFJeN9V1ZsI

TensorFlow 2.0 Complete Course - Python Neural Networks for Beginners Tutorial. (2020, March 3). YouTube. Retrieved March 23, 2022, from https://www.youtube.com/watch?v=tPYj3fFJGjk

Training and evaluation with the built-in methods. (2022, January 10). TensorFlow. Retrieved March 23, 2022, from https://www.tensorflow.org/guide/keras/train_and_evaluate