Object Detection

How to train YOLOv8 object detection on a custom dataset

February 16, 2023
10 min read

YOLOv8 is the newest addition to the YOLO family and sets new highs on the COCO benchmark. Developed by the same makers of YOLOv5, the Ultralytics team, they not only optimized the object detection algorithm but also included a highly requested feature: instance segmentation.

In this tutorial, we will take you through the steps on how to train a YOLOv8 object detector on a custom dataset using the trainYOLO platform. As an example, we will be developing a tree log detector, which can be used to accelerate the counting of tree logs. Before you start, make sure you have a trainYOLO account. If not, you can create a free account here.

Create a new project

We start by creating a new project, which will hold our images and trained models, by clicking on the green ‘plus’ icon. We will set the project visibility to Public, the name to tree detection, keep the annotation type at Bounding Boxes, and set a single category tree log. If you have multiple categories, you can add more.

To upload images to our project, go to the data tab and click on the green ‘plus’ icon. Select the images you would like to upload and click upload. In this example, we will upload 57 images containing tree logs in different configurations.

Label your images

Once we have uploaded the images, we start labeling. By clicking on an image, you enter the labeling editor. By selecting the bounding box tool in the left menu, we can draw a box around each object, in our case each tree log. Note that if you have multiple object categories, you need to set the right category using the drop-down list on top. Once finished, you can use the arrow keys on top to go to the next image or select a specific one in the carousel at the bottom.

Once we have labeled around 20 images, we will start training a first YOLOv8 version. This way, we can accelerate the labeling of the other images using model-assisted labeling

Train YOLOv8 model 

Once you have labeled enough images, you can start training your YOLOv8 model. While you can train both locally or using cloud providers like AWS or GCP, we will use our preconfigured google Colab notebooks. Therefore, we go to the model's tab and choose the YOLOv8 notebook by clicking on the green ‘plus’ icon. 

This will open our preconfigured notebook for YOLOv8 object detection. We fill in our API key (which you’ll get by clicking on your avatar next to your username) and our project name. We leave the other parameters to their default values and start the training session by clicking on “Runtime” => “Run all”. This will download your dataset, start training a model and upload the trained model to our platform. 

After training, you can see the first version added to the model’s tab.

Model-assisted labeling

We can now use our first trained model as a labeling assistant, using so-called model-assisted labeling. Each time you add a new version to your project, this latest version will be used as the labeling assistant. Instead of labeling an image from scratch, we now only need to verify or correct the model predictions. This dramatically speeds up the labeling process. 

To use the labeling assistant, click on the magic paintbrush icon in the left menu. This will load the model predictions. Next, adapt or remove unwanted bounding boxes and label the ones that are not labeled.

Next steps

Once you have labeled more images, you can train more versions and change the hyperparameters like image size, model type, etc. to optimize your model. Have fun training models!

Latest posts

Get in touch.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
By clicking “Accept All Cookies”, you agree to the storing of cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. View our Privacy Policy for more information.