I’m working with YOLOv11 for custom object detection and need to find the main training components in the Ultralytics repository.
In previous versions like YOLOv5 and YOLOv8, there were clear training files with obvious entry points. But with YOLOv11, I can’t seem to find the main training class or module that handles the core functionality.
I want to access the fundamental training mechanics like data loading, model forward passes, loss computation, and gradient updates. My goal is to add custom optimization algorithms (like Genetic Algorithm or Particle Swarm) to tune hyperparameters during training.
What I need to know:
- Is there a main training class in YOLOv11 that controls the training process?
- What’s the exact location of this component in the repo structure?
- Can the training be called programmatically or only through command line?
- What’s the best approach for adding custom hyperparameter optimization?
- Has anyone successfully modified YOLOv11’s training pipeline?
Any advice would be really helpful, especially from those who have worked with YOLOv11’s internal architecture.
Thanks!
Had the same confusion migrating from YOLOv8 to YOLOv11 last month. They restructured the training architecture pretty heavily. The main training stuff lives in ultralytics/engine/trainer.py in the BaseTrainer class - that’s where you’ll find the training loop, loss computation, and optimization. For YOLOv11, the YOLO class in ultralytics/models/yolo/detect/train.py inherits from DetectionTrainer, which inherits from BaseTrainer. You can definitely call training programmatically - I’ve been using model.train() in my scripts instead of CLI commands and it works great. For hyperparameter optimization, I’d subclass DetectionTrainer and override the optimizer_step method or use custom callbacks. YOLOv11’s callback system is pretty flexible - you can hook into different training events without messing with the core pipeline too much.
Check the ultralytics/engine/ directory - that’s where everything happens now. YOLOv11 moved to a unified engine system, so it’s different from older versions. The BaseTrainer class runs all the core training stuff and you can extend it for custom optimizers like GA or PSO. I use custom callbacks to inject my hyperparameter tuning instead of messing with the training loop directly. Way cleaner approach.
YOLOv11 uses a centralized engine system instead of standalone training scripts. Check out ultralytics/engine/trainer.py - the BaseTrainer class is your main entry point. The YOLO-specific stuff is in ultralytics/models/yolo/detect/train.py with the DetectionTrainer class. For hyperparameter optimization, I’d create a custom trainer by inheriting from DetectionTrainer. You can override methods like _setup_train or hook into the callback system at on_train_epoch_end events. The callback approach is cleaner since you don’t have to modify core files. I’ve used this to integrate custom learning rate schedules - just register callbacks that adjust hyperparameters based on validation metrics. Programmatic training works great too. Just instantiate the YOLO class and call the train method with your config. Way more flexible than CLI for custom stuff.