-
-
Notifications
You must be signed in to change notification settings - Fork 17.1k
Description
Search before asking
- I have searched the YOLOv5 issues and discussions and found no similar questions.To pick up a draggable item, press the space bar. While dragging, use the arrow keys to move the item. Press space again to drop the item in its new position, or press escape to cancel.
Question
Hi!
I am trying to evaluate my trained YOLO model on a (custom) test dataset. Yet it seems that I get very similar results for different IOU-thresholds e.g. 0.3 and 0.8. Also for threshold values of 0.1 and 0.2 precision and recall are 1. These results don't seem right.
Also, could anyone explain what influence the threshold has on the mAP value, i.e. if the threshold is larger than 0.5 shouldn't the mAP values decrease?
Thank you in advance for your help!
Below are the executed commands and outputs:
!python val.py --weights ../../content/drive/MyDrive/Masterarbeit/Class_Alldata_LISA1280_Bosch1280_DTLD1280/weights/best.pt --data coco128.yaml --iou 0.2 --img 1280 --verbose --save-hybrid
val: data=/content/yolov5/data/coco128.yaml, weights=['../../content/drive/MyDrive/Masterarbeit/Class_Alldata_LISA1280_Bosch1280_DTLD1280/weights/best.pt'], batch_size=32, imgsz=1280, conf_thres=0.001, iou_thres=0.2, task=val, device=, single_cls=False, augment=False, verbose=True, save_txt=True, save_hybrid=True, save_conf=False, save_json=False, project=runs/val, name=exp, exist_ok=False, half=False
YOLOv5 🚀 v6.0-55-g0155548 torch 1.9.0+cu111 CUDA:0 (Tesla V100-SXM2-16GB, 16160MiB)
Fusing layers...
Model Summary: 392 layers, 47047521 parameters, 0 gradients, 115.5 GFLOPs
val: Scanning '../../content/drive/MyDrive/test_images_combined/labels.cache' images and labels... 14623 found, 0 missing, 15 empty, 15 corrupted: 100% 14625/14625 [00:00<?, ?it/s]
Class Images Labels P R mAP@.5 mAP@.5:.95: 100% 457/457 [08:27<00:00, 1.11s/it]
all 14610 38674 1 1 0.995 0.995
Red 14610 14463 1 1 0.995 0.995
Yellow 14610 1437 1 1 0.995 0.995
Green 14610 20472 1 1 0.995 0.995
off 14610 2302 1 1 0.995 0.995
Speed: 0.3ms pre-process, 25.1ms inference, 1.2ms NMS per image at shape (32, 3, 1280, 1280)
Results saved to runs/val/exp4
14604 labels saved to runs/val/exp4/labels
!python val.py --weights ../../content/drive/MyDrive/Masterarbeit/Class_Alldata_LISA1280_Bosch1280_DTLD1280/weights/best.pt --data coco128.yaml --iou 0.3 --img 1280 --verbose --save-txt
val: data=/content/yolov5/data/coco128.yaml, weights=['../../content/drive/MyDrive/Masterarbeit/Class_Alldata_LISA1280_Bosch1280_DTLD1280/weights/best.pt'], batch_size=32, imgsz=1280, conf_thres=0.001, iou_thres=0.3, task=val, device=, single_cls=False, augment=False, verbose=True, save_txt=True, save_hybrid=False, save_conf=False, save_json=False, project=runs/val, name=exp, exist_ok=False, half=False
YOLOv5 🚀 v6.0-55-g0155548 torch 1.9.0+cu111 CUDA:0 (Tesla V100-SXM2-16GB, 16160MiB)
Fusing layers...
Model Summary: 392 layers, 47047521 parameters, 0 gradients, 115.5 GFLOPs
val: Scanning '../../content/drive/MyDrive/test_images_combined/labels.cache' images and labels... 14623 found, 0 missing, 15 empty, 15 corrupted: 100% 14625/14625 [00:00<?, ?it/s]
Class Images Labels P R mAP@.5 mAP@.5:.95: 100% 457/457 [08:23<00:00, 1.10s/it]
all 14610 38674 0.618 0.418 0.42 0.232
Red 14610 14463 0.681 0.502 0.522 0.271
Yellow 14610 1437 0.411 0.384 0.337 0.191
Green 14610 20472 0.71 0.636 0.627 0.341
off 14610 2302 0.67 0.149 0.195 0.125
Speed: 0.4ms pre-process, 25.1ms inference, 0.9ms NMS per image at shape (32, 3, 1280, 1280)
Results saved to runs/val/exp5
14503 labels saved to runs/val/exp5/labels
!python val.py --weights ../../content/drive/MyDrive/Masterarbeit/Class_Alldata_LISA1280_Bosch1280_DTLD1280/weights/best.pt --data coco128.yaml --iou 0.8 --img 1280 --verbose --save-txt
val: data=/content/yolov5/data/coco128.yaml, weights=['../../content/drive/MyDrive/Masterarbeit/Class_Alldata_LISA1280_Bosch1280_DTLD1280/weights/best.pt'], batch_size=32, imgsz=1280, conf_thres=0.001, iou_thres=0.8, task=val, device=, single_cls=False, augment=False, verbose=True, save_txt=True, save_hybrid=False, save_conf=False, save_json=False, project=runs/val, name=exp, exist_ok=False, half=False
YOLOv5 🚀 v6.0-55-g0155548 torch 1.9.0+cu111 CUDA:0 (Tesla V100-SXM2-16GB, 16160MiB)
Fusing layers...
Model Summary: 392 layers, 47047521 parameters, 0 gradients, 115.5 GFLOPs
val: Scanning '../../content/drive/MyDrive/test_images_combined/labels.cache' images and labels... 14623 found, 0 missing, 15 empty, 15 corrupted: 100% 14625/14625 [00:00<?, ?it/s]
Class Images Labels P R mAP@.5 mAP@.5:.95: 100% 457/457 [08:49<00:00, 1.16s/it]
all 14610 38674 0.612 0.418 0.408 0.221
Red 14610 14463 0.674 0.502 0.502 0.255
Yellow 14610 1437 0.405 0.384 0.335 0.188
Green 14610 20472 0.704 0.636 0.612 0.322
off 14610 2302 0.664 0.149 0.184 0.119
Speed: 0.4ms pre-process, 25.2ms inference, 0.9ms NMS per image at shape (32, 3, 1280, 1280)
Results saved to runs/val/exp7
14503 labels saved to runs/val/exp7/labels
Additional
No response
Activity
MrinalJain17 commentedon Nov 5, 2021
@matpy1 You've passed the
--save-hybrid
flag in the first command, which ends up creating files containing the ground truth labels first, followed by the detections. That's why you see all your metrics to be close to 1. Using--save-txt
instead should give you the expected results.I'm not sure if there is a bug when using
--save-hybrid
.github-actions commentedon Dec 6, 2021
👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.
Access additional YOLOv5 🚀 resources:
Access additional Ultralytics ⚡ resources:
Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!
Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!