Scribd Logo
Upload

Welcome to Scribd!

  • Using Trasfer Learning

    Uploaded by

    Rimsa Goperma
    20 views6 pages
    This document summarizes the process of collecting skin disease image data using web scraping and training a convolutional neural network (CNN) model for classification. Data was scraped from various websites using Beautiful Soup since publicly available datasets were limited. The CNN model was trained using transfer learning with VGG16, with the pretrained weights frozen and new convolutional layers added. The trained model was saved in h5 and JSON formats for deployment in a Django web application, where it could perform predictions by importing the model.

    Original Description:

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    This document summarizes the process of collecting skin disease image data using web scraping and training a convolutional neural network (CNN) model for classification. Data was scraped from various websites using Beautiful Soup since publicly available datasets were limited. The CNN model was trained using transfer learning with VGG16, with the pretrained weights frozen and new convolutional layers added. The trained model was saved in h5 and JSON formats for deployment in a Django web application, where it could perform predictions by importing the model.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    20 views6 pages

    Using Trasfer Learning

    Uploaded by

    Rimsa Goperma
    This document summarizes the process of collecting skin disease image data using web scraping and training a convolutional neural network (CNN) model for classification. Data was scraped from various websites using Beautiful Soup since publicly available datasets were limited. The CNN model was trained using transfer learning with VGG16, with the pretrained weights frozen and new convolutional layers added. The trained model was saved in h5 and JSON formats for deployment in a Django web application, where it could perform predictions by importing the model.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 6

    Data Collection:

    There is insufficient amount of data for skin disease like Acne, Rosacea etc. So, collected dataset for
    similar purpose like found in Kaggle was not available. Some of the data was found but it was all in the
    website which was dirty data. Hence web scraping was used for collecting data from various sources.
    Web scraping is a strategy to naturally access and concentrate a lot of data from a site, which can
    spare an enormous measure of time and exertion. For web scraping Beautiful Soup a python
    framework is used in order to collect data.

    Step 1: All the necessary library for making requests are imported.

    Step 2: The link to the website that we are aiming to extract data is given and a html parser is used
    to parse image elements.

    Step 3: The source of data is stored in a link array for downloading.


    Step 4: After successfully collecting all the links then a directory is created for storing all the jpg file.

    Step 5: We can visualize the collected data from the website.

    Similar steps is applied for collecting data of other classes like Rosacea, Eczema, and Sun Burn.
    Training Model:

    Importing Libraries: Initially all the necessary libraries like os, NumPy, Keras, VGG16, OpenCV are
    imported.

    Data augmentation: Here the path to training and validation dataset is given. Additionally various
    parameters needed during training process is also configured. This way you can reduce overfitting
    your classifier. This way we can generate different sub samples of images. This method is useful for
    tackling problems like overfitting as it increases the volume of the dataset. For small amount of data
    this technique can bring a lot of merits. The validation data is not augmented it is only rescaled.
    Loading VGG: VGG16 is a pretrained model we have to download the model and tweak many of its
    parameter for our purpose. The original input shape of the model is (224, 224, and 3) but it is change
    to (128, 128, 3). Then the shape of the final layer is printed. This is useful for performing transfer
    learning ahead.

    Transfer Learning: For transfer learning we can use various technique like training the whole model
    or only training half of the layer and freezing others. But here we are freezing all the layers of the
    VGG model and adding a new CNN layers from the base. This is very useful as the information gained
    by original model which was trained in ImageNet is not lost. This helps to improve performance of
    our model even more.

    The model is then compiled. The RMSProp optimizer is used.


    The model is then fit with validation set and training set. The model is trained for 50 epochs.

    Model Deployment:

    The weights of the trained model is then saved in h5 file format.

    After this the h5 file is converted into JSON format. This step is necessary for loading the model in
    our Django Application. The JSON format is easier to load as a rest API. The following code converts
    the model to JSON format.

    The screenshot below shows how the model is loaded in Django Application. After loading the model
    the prediction process can be applied by importing the model in view.py file.

    You might also like