Skip to content

Implementations instrumentation framework for OpenCV performance test system; #7101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 18, 2016
Merged

Implementations instrumentation framework for OpenCV performance test system; #7101

merged 1 commit into from
Aug 18, 2016
+790 −32

Conversation

lupustr3
Copy link
Contributor

This pullrequest changes

Instrumentation framework for OpenCV performance test system. It was developed to:

  • track implementations execution inside OpenCV functions with call tree output to test logs
  • track implementations execution time weight in the pipeline

Framework can be enabled with ENABLE_INSTRUMENTATION define in CMake and disabled by default. In disabled state it doesn't consume resources.

Example instrumentation was implemented for cv::meanStdDev function:

Output example for IPP:

[ RUN      ] Size_MatType_meanStdDev.meanStdDev/6
[ PERFSTAT ]    (samples = 25, mean = 0.10, median = 0.10, stddev = 0.00 (0.5%))
[ VALUE    ]    (1920x1080, 8UC1)
[ TRACE    ]
ROOT
\---cv::meanStdDev - C:25 /*entries counter*/ T:2.3906ms /*block execution time*/
    \---cv::ipp_meanStdDev<W_IPP> - C:25 T:2.3269ms L:97% /*weight in local block*/ G:97% /*weight in entire pipeline*/
        \---ippiMean_StdDev_C1R<F_IPP> - C:25 T:2.2700ms L:98% G:95%

IPP weight: 95.0% /*total implementation functions execution weight across all entries*/
OPENCL weight: 0.0%
[/TRACE    ]
[       OK ] Size_MatType_meanStdDev.meanStdDev/6 (77 ms)

Output example for OpenCL:

[ RUN      ] OCL_MeanStdDevFixture_MeanStdDev.MeanStdDev/7
[ PERFSTAT ]    (samples = 25, mean = 0.23, median = 0.23, stddev = 0.00 (1.8%))
[ VALUE    ]    (1280x720, 32FC1)
[ TRACE    ]
ROOT
\---cv::meanStdDev - C:25 T:5.8007ms
    \---cv::ocl_meanStdDev<W_OCL> - C:25 T:5.7412ms L:99% G:99%
        \---k.run<F_OCL> - C:25 T:3.5630ms L:62% G:61%

IPP weight: 0.0%
OPENCL weight: 61.4%
[/TRACE    ]
[       OK ] OCL_MeanStdDevFixture_MeanStdDev.MeanStdDev/7 (36 ms)

@lupustr3 lupustr3 force-pushed the pvlasov/instrumentation branch from 183195c to 19811a7 Compare August 12, 2016 07:05
@alalek
Copy link
Member

alalek commented Aug 12, 2016

Awesome useful feature!

alalek
alalek reviewed Aug 12, 2016
View reviewed changes
modules/ts/src/ts_perf.cpp
}
}

static double calcLocWeight(cv::instr::InstrNode *pNode)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calcLocWeight -> calcLocalWeight ?

Same note for calcGlobWeight -> calcGlobalWeight

@lupustr3 lupustr3 force-pushed the pvlasov/instrumentation branch 2 times, most recently from 380e544 to 7a80926 Compare August 12, 2016 12:08
@lupustr3
Copy link
Contributor Author

"--perf_instrument" command line switch was added to ps

@lupustr3 lupustr3 force-pushed the pvlasov/instrumentation branch 2 times, most recently from 950dc9e to f689b9c Compare August 16, 2016 06:43
alalek
alalek reviewed Aug 16, 2016
View reviewed changes
modules/core/src/system.cpp
Node<OBJECT>::Node()
{
m_pParent = 0;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, implementation of template classes should be placed in headers (.hpp or .impl.hpp)

@lupustr3 lupustr3 force-pushed the pvlasov/instrumentation branch from f689b9c to 2a4efec Compare August 16, 2016 14:32
@lupustr3
Copy link
Contributor Author

  • Parallel regions handling was redesigned and improved.
  • Total time accumulators were replaced with running mean time accumulators to decrease overhead of threads timing synchronization in parallel regions
  • Changes according to notes above.

@alalek
Copy link
Member

alalek commented Aug 16, 2016

Great update! Looks good.

BTW, What do you think about these macro changes? Are there any expected problems?

@lupustr3
Copy link
Contributor Author

lupustr3 commented Aug 17, 2016 •
edited
Loading

I think it should work the same.

Do you want me to integrate these changes into my commit?

@lupustr3 lupustr3 force-pushed the pvlasov/instrumentation branch from 2a4efec to 9fec107 Compare August 17, 2016 13:22
@lupustr3
Copy link
Contributor Author

New macro interface

@lupustr3 lupustr3 force-pushed the pvlasov/instrumentation branch from 9fec107 to a462d76 Compare August 17, 2016 13:26
@alalek
Copy link
Member

alalek commented Aug 18, 2016

👍

@hrnr hrnr mentioned this pull request Aug 18, 2016
Merged
24 tasks
@opencv-pushbot opencv-pushbot merged commit a462d76 into opencv:master Aug 18, 2016
opencv-pushbot pushed a commit that referenced this pull request Aug 18, 2016
@alalek
Merge pull request #7101 from lupustr3:pvlasov/instrumentation

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode
93e5947
@PkLab PkLab mentioned this pull request Sep 12, 2016
Open
@seiko2plus seiko2plus mentioned this pull request Jan 7, 2021
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@alalek alalek

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@lupustr3 @alalek @opencv-pushbot