Skip to content

val.py results #5508

@matpy1

Description

@matpy1

Search before asking

  • I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

Hi!
I am trying to evaluate my trained YOLO model on a (custom) test dataset. Yet it seems that I get very similar results for different IOU-thresholds e.g. 0.3 and 0.8. Also for threshold values of 0.1 and 0.2 precision and recall are 1. These results don't seem right.
Also, could anyone explain what influence the threshold has on the mAP value, i.e. if the threshold is larger than 0.5 shouldn't the mAP values decrease?

Thank you in advance for your help!

Below are the executed commands and outputs:

!python val.py --weights ../../content/drive/MyDrive/Masterarbeit/Class_Alldata_LISA1280_Bosch1280_DTLD1280/weights/best.pt --data coco128.yaml --iou 0.2 --img 1280 --verbose --save-hybrid

val: data=/content/yolov5/data/coco128.yaml, weights=['../../content/drive/MyDrive/Masterarbeit/Class_Alldata_LISA1280_Bosch1280_DTLD1280/weights/best.pt'], batch_size=32, imgsz=1280, conf_thres=0.001, iou_thres=0.2, task=val, device=, single_cls=False, augment=False, verbose=True, save_txt=True, save_hybrid=True, save_conf=False, save_json=False, project=runs/val, name=exp, exist_ok=False, half=False
YOLOv5 🚀 v6.0-55-g0155548 torch 1.9.0+cu111 CUDA:0 (Tesla V100-SXM2-16GB, 16160MiB)

Fusing layers...
Model Summary: 392 layers, 47047521 parameters, 0 gradients, 115.5 GFLOPs
val: Scanning '../../content/drive/MyDrive/test_images_combined/labels.cache' images and labels... 14623 found, 0 missing, 15 empty, 15 corrupted: 100% 14625/14625 [00:00<?, ?it/s]

           Class     Images     Labels          P          R     mAP@.5 mAP@.5:.95: 100% 457/457 [08:27<00:00,  1.11s/it]
             all      14610      38674          1          1      0.995      0.995
             Red      14610      14463          1          1      0.995      0.995
          Yellow      14610       1437          1          1      0.995      0.995
           Green      14610      20472          1          1      0.995      0.995
             off      14610       2302          1          1      0.995      0.995

Speed: 0.3ms pre-process, 25.1ms inference, 1.2ms NMS per image at shape (32, 3, 1280, 1280)
Results saved to runs/val/exp4
14604 labels saved to runs/val/exp4/labels

!python val.py --weights ../../content/drive/MyDrive/Masterarbeit/Class_Alldata_LISA1280_Bosch1280_DTLD1280/weights/best.pt --data coco128.yaml --iou 0.3 --img 1280 --verbose --save-txt

val: data=/content/yolov5/data/coco128.yaml, weights=['../../content/drive/MyDrive/Masterarbeit/Class_Alldata_LISA1280_Bosch1280_DTLD1280/weights/best.pt'], batch_size=32, imgsz=1280, conf_thres=0.001, iou_thres=0.3, task=val, device=, single_cls=False, augment=False, verbose=True, save_txt=True, save_hybrid=False, save_conf=False, save_json=False, project=runs/val, name=exp, exist_ok=False, half=False
YOLOv5 🚀 v6.0-55-g0155548 torch 1.9.0+cu111 CUDA:0 (Tesla V100-SXM2-16GB, 16160MiB)

Fusing layers...
Model Summary: 392 layers, 47047521 parameters, 0 gradients, 115.5 GFLOPs
val: Scanning '../../content/drive/MyDrive/test_images_combined/labels.cache' images and labels... 14623 found, 0 missing, 15 empty, 15 corrupted: 100% 14625/14625 [00:00<?, ?it/s]

           Class     Images     Labels          P          R     mAP@.5 mAP@.5:.95: 100% 457/457 [08:23<00:00,  1.10s/it]
             all      14610      38674      0.618      0.418       0.42      0.232
             Red      14610      14463      0.681      0.502      0.522      0.271
          Yellow      14610       1437      0.411      0.384      0.337      0.191
           Green      14610      20472       0.71      0.636      0.627      0.341
             off      14610       2302       0.67      0.149      0.195      0.125

Speed: 0.4ms pre-process, 25.1ms inference, 0.9ms NMS per image at shape (32, 3, 1280, 1280)
Results saved to runs/val/exp5
14503 labels saved to runs/val/exp5/labels

!python val.py --weights ../../content/drive/MyDrive/Masterarbeit/Class_Alldata_LISA1280_Bosch1280_DTLD1280/weights/best.pt --data coco128.yaml --iou 0.8 --img 1280 --verbose --save-txt

val: data=/content/yolov5/data/coco128.yaml, weights=['../../content/drive/MyDrive/Masterarbeit/Class_Alldata_LISA1280_Bosch1280_DTLD1280/weights/best.pt'], batch_size=32, imgsz=1280, conf_thres=0.001, iou_thres=0.8, task=val, device=, single_cls=False, augment=False, verbose=True, save_txt=True, save_hybrid=False, save_conf=False, save_json=False, project=runs/val, name=exp, exist_ok=False, half=False
YOLOv5 🚀 v6.0-55-g0155548 torch 1.9.0+cu111 CUDA:0 (Tesla V100-SXM2-16GB, 16160MiB)

Fusing layers...
Model Summary: 392 layers, 47047521 parameters, 0 gradients, 115.5 GFLOPs
val: Scanning '../../content/drive/MyDrive/test_images_combined/labels.cache' images and labels... 14623 found, 0 missing, 15 empty, 15 corrupted: 100% 14625/14625 [00:00<?, ?it/s]

           Class     Images     Labels          P          R     mAP@.5 mAP@.5:.95: 100% 457/457 [08:49<00:00,  1.16s/it]
             all      14610      38674      0.612      0.418      0.408      0.221
             Red      14610      14463      0.674      0.502      0.502      0.255
          Yellow      14610       1437      0.405      0.384      0.335      0.188
           Green      14610      20472      0.704      0.636      0.612      0.322
             off      14610       2302      0.664      0.149      0.184      0.119

Speed: 0.4ms pre-process, 25.2ms inference, 0.9ms NMS per image at shape (32, 3, 1280, 1280)
Results saved to runs/val/exp7
14503 labels saved to runs/val/exp7/labels

Additional

No response

Activity

MrinalJain17

MrinalJain17 commented on Nov 5, 2021

@MrinalJain17
Contributor

@matpy1 You've passed the --save-hybrid flag in the first command, which ends up creating files containing the ground truth labels first, followed by the detections. That's why you see all your metrics to be close to 1. Using --save-txt instead should give you the expected results.

I'm not sure if there is a bug when using --save-hybrid.

github-actions

github-actions commented on Dec 6, 2021

@github-actions
Contributor

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

Access additional Ultralytics ⚡ resources:

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    StaleStale and schedule for closing soonquestionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @MrinalJain17@matpy1

        Issue actions

          val.py results · Issue #5508 · ultralytics/yolov5