Performing data augmentation(20190819)

Th following code is for making the input of transformations to go through the augmenter functions. I have called several classes defined by pytorch APIs.

The UML below shows how specific functions are called. Here every function intializes the transform and goes through the aug function.

Next the path is accessed through glob. Glob is a path calling function.

Image open pulls the image and applies transform to each.

In the end the image length loads all the images of particular length and is shown as the output after iteration is run.

Following is the link to my code.

Using augmenter functions(20190819)

In the last post I mentioned about using some augmenters. These functions will help in increasing my dataset which will result in a proper trained model.

The code with augmenter functions can be seen below:

After defining augmenters, the next step is to iterate and show the output.

This code gives me run time error. (previous batch_size=24)

But if I change my batch_size=1 the dataloader code containing augmenters works fine.

Adding more augmenters(20190816)

In the last post I mentioned about installing albumnetations and using three augmenter functions.

But I think functions like IAAEmboss(), Transpose() will also be helpful.

I am trying to use IAAEmbos, it embosses the input image and overlays the result with the original image. The parameters for this function will be defined later.

The dataloader code after applying these functions looks like this:

This is just the part of the code, complete code can be viewed through this link:

Importing Albumentation and using augmenters(20190816)

In order to successfully import library albumentations, I first installed it.

Albumentation is a pytorch library for the augmentation of data.

After installing and importing the albumentations, in the next step I will perform 3 data augmentation functions in my dataloader code.

The link for my dataloader code is: https://postechackr-my.sharepoint.com/personal/juhpark_postech_ac_kr/_layouts/15/onedrive.aspx?id=%2Fpersonal%2Fjuhpark%5Fpostech%5Fac%5Fkr%2FDocuments%2FPDI%20Lab%2FMisha%2F20190813%20DATA%20PREPARATION%2FPart%5F3%5FDATALOADER%5FSkeleton%5Fcode%2820190813%29%2Epy&parent=%2Fpersonal%2Fjuhpark%5Fpostech%5Fac%5Fkr%2FDocuments%2FPDI%20Lab%2FMisha%2F20190813%20DATA%20PREPARATION

Even though I haven’t applied yet the following 3 functions will be added to the code described above.

CLAHE stands for Contrast-limited adaptive histogram equalization. It is customized transform.

IAASharpen() is an augmenter that sharpens images and overlays the result with the original image. Right now I am not varying the values for this function, it will be done later.

Image Augmentation(20190815)

Today I am trying to use augmentation for the images, I hope this works and some functions result in increased data size.

In the code below I am trying to use albumentations, this is a fast image augmentation library.

Even though I am importing all the possible functions but later will use only few of them.

Most probably I will use 7-8 functions some of which are random brightness, random rotate etc.

I received module not found error while importing, therefore I need to install some stuff.

UML for data loader, image show and data splitting code(20190813)

Following code performs data loading.

I drew an UML of this code to explain it better, I hope this helps.

Excuse me for bad handwriting.

The next code is for showing the grid of images. The UML for this code is just below it.

The next code was executed by me today, it is for splitting the data into train and validation set. The UML is below the code.

Splitting the dataset into train and validation sets(20190813)

I tried splitting my data into train and validation set today.

I tried to keep my train set larger that’s why the integer value assigned is 0.7.

The validation set will be the train set subtracted by the total set as can be inferred from the code below.

I did random splitting as it can be seen in the image below –

The code ran successfully and divided the dataset as desired. The output can be seen below –

Any value less than 0.7 will result in train and validation sets to be approximately equal. The output is shown below –

Following is the link for the code above:

Clipping data size and loading large number of images successfully(20190812)

The following code defines transforms and performs augmentation of images.

The UML below describes the working of the code.

I made several changes and applied transforms to reduce the size of my dataset. The screen shot below shows the complete data loader code which ran successfully.

The code below shows the next step where I defined batch_size = 25 and this time it ran successfully.

As it can be inferred the output shows a grid of 25 images which means the transformation was successful.

Since all the codes are working fine now the next step is to import and split our data into train and validation.

Following is the link to my one drive file –

Defining transforms in data loader (20190812)

In the last blog post I mentioned about how my data loader is only working for batch_size=1 and can load only one image.

The output can be seen below:

I tried defining the transforms, transforms.compose will allow us to perform multiple transforms like resizing and cropping of the images.

While defining transforms, the following syntax error occurred.

In the current code I commented the normalize section, therefore no normalization will occur if code runs correctly.

And with normalization the error occurs again.

The resize function in the above code will resize the images so the shortest side has a length of 255 pixels. The other side will be scaled to maintain the aspect ratio of the image.

The CenterCrop function will crop the center of the image so it is a 224 by 224 pixels square image.

I am trying to perform these two steps successfully so that all the images fed to AI model have the same size.

Preparing dataset for data loader (20190809)

In the last blog I was stuck in data loader part where following error occurred.

I realized there might be a problem due to batch size or images are too big.

Therefore I did some research and when I assigned batch_size=1, the code worked and gave an output of single image. From this I can infer that the images are way bigger in size and only one image can be processed by the data loader.

Previously I tried with batch_size=24,20,18,15 etc. which all resulted in error. Currently batch_size=1 is only supported.

If batch_size=1 everything runs well and my data loader loads a single image which is understandable because of image size. The output is shown below:

I think of applying some transformation in my data loader code to overcome this error,

Create your website at WordPress.com
Get started