Skip to content

ruy optimized library enabling for tf-lite #39812

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
peter197321 opened this issue May 23, 2020 · 4 comments
Closed

ruy optimized library enabling for tf-lite #39812

peter197321 opened this issue May 23, 2020 · 4 comments
Assignees
@talumbau
Labels
comp:lite TF Lite related issues type:support Support issues

Comments

@peter197321
Copy link

peter197321 commented May 23, 2020 •
edited
Loading

Per blog What’s new in TensorFlow Lite there's supposed to be developed an optimized matrix multiplication library (ruy).

How to check if this (ruy) optimized library is enabled during a build (compilation) from the source or how to check or make sure it's either enabled or disabled?
@peter197321 peter197321 added the type:build/install Build and install issues label May 23, 2020
@Saduf2019
Copy link
Contributor

@peter197321
Please refer to this link link1 and let us know if this helps.

@Saduf2019 Saduf2019 added type:support Support issues stat:awaiting response Status - Awaiting response from author and removed type:build/install Build and install issues labels May 25, 2020
@peter197321
Copy link
Author

peter197321 commented May 26, 2020 •
edited
Loading

Unfortunately, it does not really help much :( I am looking for a way to check if the lib has been compiled with "ruy" option (a version of ruy) and want to check if this option has been applied (enabled) during a build process. I guess this requires to build particularly library with option -frecord-gcc-switches

I applied readelf -a to binary file used where I found some versions info

snippet :

Version needs section '.gnu.version_r' contains 9 entries:
 Addr: 0x00000000002eba48  Offset: 0x2eba48  Link: 4 (.dynstr)
  000000: Version: 1  File: libgcc_s.so.1  Cnt: 1
  0x0010:   Name: GCC_3.0  Flags: none  Version: 24
  0x0020: Version: 1  File: ld-linux-x86-64.so.2  Cnt: 1
  0x0030:   Name: GLIBC_2.3  Flags: none  Version: 23
  0x0040: Version: 1  File: libcudart.so.10.0  Cnt: 1
  0x0050:   Name: libcudart.so.10.0  Flags: none  Version: 15
  0x0060: Version: 1  File: libm.so.6  Cnt: 1
  0x0070:   Name: GLIBC_2.2.5  Flags: none  Version: 14
  0x0080: Version: 1  File: libpthread.so.0  Cnt: 1
  0x0090:   Name: GLIBC_2.2.5  Flags: none  Version: 12
  0x00a0: Version: 1  File: libstdc++.so.6  Cnt: 14
  0x00b0:   Name: GLIBCXX_3.4.14  Flags: none  Version: 29
  0x00c0:   Name: CXXABI_1.3.8  Flags: none  Version: 27
  0x00d0:   Name: GLIBCXX_3.4.17  Flags: none  Version: 26
  0x00e0:   Name: GLIBCXX_3.4.19  Flags: none  Version: 25
  0x00f0:   Name: CXXABI_1.3.5  Flags: none  Version: 22
  0x0100:   Name: CXXABI_1.3.7  Flags: none  Version: 21
  0x0110:   Name: GLIBCXX_3.4.15  Flags: none  Version: 17
  0x0120:   Name: GLIBCXX_3.4.18  Flags: none  Version: 16
  0x0130:   Name: GLIBCXX_3.4.9  Flags: none  Version: 13
  0x0140:   Name: GLIBCXX_3.4.11  Flags: none  Version: 11
  0x0150:   Name: CXXABI_1.3  Flags: none  Version: 10
  0x0160:   Name: GLIBCXX_3.4.20  Flags: none  Version: 9
  0x0170:   Name: GLIBCXX_3.4  Flags: none  Version: 8
  0x0180:   Name: GLIBCXX_3.4.21  Flags: none  Version: 6
  0x0190: Version: 1  File: libdl.so.2  Cnt: 1
  0x01a0:   Name: GLIBC_2.2.5  Flags: none  Version: 5
  0x01b0: Version: 1  File: libcudnn.so.7  Cnt: 1
  0x01c0:   Name: libcudnn.so.7  Flags: none  Version: 4
  0x01d0: Version: 1  File: libc.so.6  Cnt: 6
  0x01e0:   Name: GLIBC_2.7  Flags: none  Version: 28
  0x01f0:   Name: GLIBC_2.3.4  Flags: none  Version: 20
  0x0200:   Name: GLIBC_2.4  Flags: none  Version: 19
  0x0210:   Name: GLIBC_2.6  Flags: none  Version: 18
  0x0220:   Name: GLIBC_2.2.5  Flags: none  Version: 7
  0x0230:   Name: GLIBC_2.11  Flags: none  Version: 3

So what either would see if ruy used or other methods to check with?
What if I use a python - which library to check?

anyway - I'll investigate more :)

@Saduf2019 Saduf2019 added comp:lite TF Lite related issues and removed stat:awaiting response Status - Awaiting response from author labels May 27, 2020
@talumbau talumbau self-assigned this May 27, 2020
@talumbau
Copy link
Member

Hi,

Hopefully I can help you but I just want to make sure I understand the case you are interested in. Please add additional clarification if my answers below don't help. My understanding is that you are concerned with enabling/detecting enablement of Ruy in (at least) one of the following scenarios:

  1. Create a build of TF Lite that has Ruy enabled.

Answer: with a bazel build, add the command line flag --define=tflite_with_ruy=true to the bazel build command. This is already the default for ARM builds, but it does no harm to add it. For Makefile-based builds, ensure that -DTFLITE_WITH_RUY is added to CXXFLAGS, as shown here:

tensorflow/tensorflow/lite/tools/make/Makefile

Line 184 in 5802195

CXXFLAGS += -DTFLITE_WITH_RUY

  1. You have a TFLite binary from somewhere and wish to inspect it to see if Ruy is enabled without executing the binary.
    Answer: there's no supported way to do this, but one could imagine a few ideas.

  2. You have the TF Lite library linked to your application and you wish to, at run time, determine if Ruy is being used for matrix multiplication.
    Answer: there is also no supported way to check here, i.e. a flag of some kind, but you can always compile with -DRUY_PROFILER and then run your program to make sure you get Ruy for the expected GEMM operations. The output at the end of program execution will tell you which GEMMs were done with Ruy (with size info) as well as what Ruy path was selected and how much time was spent on various parts of the GEMM operation.

Does that help?

@jvishnuvardhan jvishnuvardhan removed their assignment May 27, 2020
@peter197321
Copy link
Author

Thank you.
Is it apply for the Cross-compile build of TensorFlow Lite for ARM64 board?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@talumbau talumbau

Labels
comp:lite TF Lite related issues type:support Support issues
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@talumbau @jvishnuvardhan @peter197321 @Saduf2019