YOLOv4: A step-by-step guide for Custom Data Preparation with Code
post-template-default,single,single-post,postid-16199,single-format-standard,bridge-core-1.0.5,ajax_fade,page_not_loaded,,qode_grid_1300,qode-content-sidebar-responsive,qode-theme-ver-18.1,qode-theme-bridge,qode_advanced_footer_responsive_1000,bridge,wpb-js-composer js-comp-ver-6.0.2,vc_responsive

YOLOv4: A step-by-step guide for Custom Data Preparation with Code

yolov4 guide

YOLOv4: A step-by-step guide for Custom Data Preparation with Code

You only look once or YOLO is a state of the art object detection algorithm. It can be found today in real-time detection and monitoring systems like CCTV cameras and autonomous driving vehicles. It’s famous for being very accurate and fast at the same time. YOLOv4 is one of the latest versions of the YOLO family. This tutorial will go over how to prepare data in YOLOv4 format from scratch and how to train the model.

The first version of YOLO was released in 2015 by Joseph Redmon et al. You can find the original paper to YOLO at You Only Look Once: Unified, Real-Time Object Detection.

You can use my COLAB Notebook and my preprocessed train and test data or follow this tutorial.


YOLOv4: Data Preparation


The most crucial step in any deep learning task is data preparation. There is a general rule that garbage in == garbage out. It is also the most time-taking as well since we want to ensure good images and correct annotations.


Data Collection


You can find many free open datasets in Yolo format online, but for this tutorial, we’ll create one from scratch. We will create a custom traffic dataset for 5 classes (Car, Person, Number Plate, Bicycle, and Motorcycle). Open Images Dataset V6 is a free resource for gathering dataset, and OIDv4_ToolKit is a toolkit we use to download the dataset. Click on the OIDv4 toolkit link I have given and download it from the Github Repo. After downloading, open your command line and move into the directory using the cd command.


Type the following command to install required python packages.

pip install -r requirements.txt


Then to download the required dataset, type the following in your command line window.



python main.py downloader --classes Car Bicycle Person Vehicle_registration_number 
Motorcycle --type_csv train --multiclasses 1 --limit 50



python main.py downloader --classes Car Bicycle Person Vehicle_registration_number 
Motorcycle --type_csv validation --multiclasses 1 --limit 5


If there is space between words like “Vehicle registration number”, you have to write it together as Vehicle_registration_number. When you enter them, the toolkit will ask to download .csv files for the first time. Enter Y to download them. After the .csv, files the toolkit will download the images and their bounding boxes. 


Once all of this is done, you can find the downloaded dataset at 



If you open the train and validation folder, you will see that there is a “Labels” folder inside them as well. It contains bounding boxes for the object in the images we downloaded. But we can not use them since these are annotations as one object type per image. It means that if an image contains “number plates” and “cars”, the annotations will have bounding boxes for only cars or only number plates. But we need annotations for every object in the image. This can be better understood by the following diagram.


What we have vs How we need it

What we have vs How we need it


Yolo is trained better when it sees lots of information in one image, so we need to change it into the new format. For this remove the Labels folder from the “train” and “validation” folders.


Data Preparation


To prepare the dataset, we will use LabelImg (Installation procedure explained in the Github repo). It is a free open source Image annotator that we can use to create annotations in YOLOv4 format. 

Open LabelImg and open the location folder. Press “w” and make bounding boxes around objects and label them. After that, save the file. But make sure it is in .txt format and it is being saved in the same folder as the images.



Source:Open Images Dataset V6


Once you are done with the annotations, cut the file called “classes.txt” in the folder and save it somewhere safe. Because we will need to afterward. Rename the folder containing training images as “obj” and validation images as “test”. Zip them separately and upload them to your google drive.




We will use Google COLAB to train our model (if you have not used Google COLAB before, check our blog for Google COLAB). Click on this link to open the Notebook

Click on Copy to Drive to save a copy to your own Drive.


yolov4: Copy to Drive

yolov4: Copy to Drive


To connect to a GPU, goto Edit> Notebook settings> Hardware accelerator > GPU


yolov4: Notebook Settings in Google Colab

yolov4: Notebook Settings in Google Colab


Clone the Darknet Github repo

# clone darknet repo
!git clone https://github.com/AlexeyAB/darknet


To tell YOLOv4 to use the GPU and OpenCV

# change makefile to have GPU and OPENCV enabled
%cd darknet
!sed -i 's/OPENCV=0/OPENCV=1/' Makefile
!sed -i 's/GPU=0/GPU=1/' Makefile
!sed -i 's/CUDNN=0/CUDNN=1/' Makefile
!sed -i 's/CUDNN_HALF=0/CUDNN_HALF=1/' Makefile

Verify you have cuda

# verify CUDA
!/usr/local/cuda/bin/nvcc --version


You should get the following output

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243


Run the make file



Download YOLOv4 weights file

!wget https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal


Run this helper function

import cv2
import matplotlib.pyplot as plt
%matplotlib inline
def imShow(path):
  image = cv2.imread(path)
  height, width = image.shape[:2]
  resized_image = cv2.resize(image,(3*width, 3*height), interpolation = cv2.INTER_CUBIC)
  fig = plt.gcf()
  fig.set_size_inches(18, 10)
  plt.imshow(cv2.cvtColor(resized_image, cv2.COLOR_BGR2RGB))

If these runs and predictions for data/person.jpg are shown then, everything is good to go. Now we can start to train our model

!./darknet detector test cfg/coco.data cfg/yolov4.cfg yolov4.weights data/person.jpg

yolov4: person, horse, dog                             Source:Open Images Dataset V6

At the moment our link to drive is 

/content/drive/My Drive/


We can simplify this to /mydrive/ by

!ln -s /content/drive/My\ Drive/ /mydrive
!ls /mydrive


Now copy the obj.zip and the test.zip into your virtual environment

# %cd ..
!cp -r '/mydrive/data/obj.zip' /content
!cp -r '/mydrive/data/test.zip' /content


And unzip them to content>darknet>data

!unzip '/content/obj.zip' -d '/content/darknet/data'
!unzip '/content/test.zip' -d '/content/darknet/data'


Go to the OIDv4_ToolKit-master folder on your computer and upload generate_train.py and generate_test.py into /content/darknet


Then run the following commands

!python generate_train.py
!python generate_test.py


Verify that you have train.txt and test.txt in your /content/darknet/data folder.

!ls data/


yolov4: verify test & train

yolov4: verify test & train


Download the weights of the convolutional layers.



We need to make two files in /content/darknet/data. Namely obj.data and obj.names

obj.data should have the following contents


yolov4: Obj.data

yolov4: Obj.data


After that, go to your drive and create a folder called backup. YOLOv4 saves model weights after every 100 iterations and after 1000, it saves after every 1000 iterations. If training stops for some reason, you can restart it from the last saved weights file.

obj.names should have the same contents as the classes.txt that you saved during the Data Preparation phase. In the same order, this is how mine looks:


yolov4: obj.names

yolov4: obj.names


Finally, you need to change the contents of the config.

Go to /content/darknet/cfg/ and open  yolov4-custom.cfg


Make the following changes:

max_batches = 10000 (num_classes*2000 but if classes are less then or equal to 3 put 
max_batches = 6000)
width = 416 (has to be multiple of 32, increase height and width will increase accuracy 
but training speed will slow down).
height = 416 (has to be multiple of 32).

steps = 8000, 9000 (80% of max_batches), (90% of max_batches) 


Finally scroll done the file and find classes and filters (they are in three different 
locations so change all).
classes = 5
filters = 30 ( (num_classes + 5) * 3 )
Save the file after making all these changes.


And that’s it! Your model is ready to train. Run this to train:

# %%capture
!./darknet detector train data/obj.data cfg/yolov4-custom.cfg yolov4.conv.137 -dont_show -map

If your notebook starts to crash, just uncomment the %%capture line.

In Order to start off training from where you saved your weights, use this:

!./darknet detector train data/obj.data cfg/yolov4-custom.cfg /mydrive/backup
/yolov4-obj_last.weights -dont_show




As soon as you start the training this text will start to appear. 

It will show the iteration and the current loss of the model. The loss will go down fast at the beginning but it will slow down as iterations increase.


yolov4: loss

yolov4: loss


You can also observe the loss using this graph below. It is by default saved in the darknet directory. 


yolov4: loss

yolov4: loss graph


Let the model run for a couple of hours until the graph curve starts to flatten out.


YOLOv4: Make Predictions


To make predictions, first, run this to change the yolov4-custom.cfg 

%cd cfg
!sed -i 's/batch=64/batch=1/' yolov4-custom.cfg
!sed -i 's/subdivisions=16/subdivisions=1/' yolov4-custom.cfg
%cd ..


Finally, run this command to run predictions:

!./darknet detector test data/obj.data cfg/yolov4-custom.cfg /mydrive/backup
/yolov4-custom_best.weights '/content/test_image.jpg' -thresh 0.6


Seems pretty accurate!!!


yolov4: heavy bike

yolov4: heavy bike

yolov4: person & cycle

yolov4: person & cycle


yolov4: car

yolov4: car


Ahsan Mukhtar

Ahsan is a Machine Learning Research Engineer at Techylem, working diligently to explore new dimensions of Machine Learning and Artificial Intelligence. He is gaming passionate and does gaming in his leisure time.