Skip to content

ruy optimized library enabling for tf-lite #39812

New issue
New issue
Closed
Closed
ruy optimized library enabling for tf-lite#39812
Assignees
talumbau
Labels
comp:liteTF Lite related issuestype:supportSupport issues
@peter197321

Description

@peter197321
peter197321
opened on May 23, 2020 · edited by peter197321

Per blog What’s new in TensorFlow Lite there's supposed to be developed an optimized matrix multiplication library (ruy).

How to check if this (ruy) optimized library is enabled during a build (compilation) from the source or how to check or make sure it's either enabled or disabled?

Activity

peter197321
added
type:build/installBuild and install issues
on May 23, 2020
google-ml-butler
assigned
Saduf2019
on May 23, 2020
Saduf2019

Saduf2019 commented on May 25, 2020

@Saduf2019
Saduf2019
on May 25, 2020
Contributor

@peter197321
Please refer to this link link1 and let us know if this helps.

Saduf2019
added
type:supportSupport issues
stat:awaiting responseStatus - Awaiting response from author
and removed
type:build/installBuild and install issues
on May 25, 2020
peter197321

peter197321 commented on May 26, 2020

@peter197321
peter197321
on May 26, 2020 · edited by peter197321
Author

Unfortunately, it does not really help much :( I am looking for a way to check if the lib has been compiled with "ruy" option (a version of ruy) and want to check if this option has been applied (enabled) during a build process. I guess this requires to build particularly library with option -frecord-gcc-switches

I applied readelf -a to binary file used where I found some versions info

snippet :

Version needs section '.gnu.version_r' contains 9 entries:
 Addr: 0x00000000002eba48  Offset: 0x2eba48  Link: 4 (.dynstr)
  000000: Version: 1  File: libgcc_s.so.1  Cnt: 1
  0x0010:   Name: GCC_3.0  Flags: none  Version: 24
  0x0020: Version: 1  File: ld-linux-x86-64.so.2  Cnt: 1
  0x0030:   Name: GLIBC_2.3  Flags: none  Version: 23
  0x0040: Version: 1  File: libcudart.so.10.0  Cnt: 1
  0x0050:   Name: libcudart.so.10.0  Flags: none  Version: 15
  0x0060: Version: 1  File: libm.so.6  Cnt: 1
  0x0070:   Name: GLIBC_2.2.5  Flags: none  Version: 14
  0x0080: Version: 1  File: libpthread.so.0  Cnt: 1
  0x0090:   Name: GLIBC_2.2.5  Flags: none  Version: 12
  0x00a0: Version: 1  File: libstdc++.so.6  Cnt: 14
  0x00b0:   Name: GLIBCXX_3.4.14  Flags: none  Version: 29
  0x00c0:   Name: CXXABI_1.3.8  Flags: none  Version: 27
  0x00d0:   Name: GLIBCXX_3.4.17  Flags: none  Version: 26
  0x00e0:   Name: GLIBCXX_3.4.19  Flags: none  Version: 25
  0x00f0:   Name: CXXABI_1.3.5  Flags: none  Version: 22
  0x0100:   Name: CXXABI_1.3.7  Flags: none  Version: 21
  0x0110:   Name: GLIBCXX_3.4.15  Flags: none  Version: 17
  0x0120:   Name: GLIBCXX_3.4.18  Flags: none  Version: 16
  0x0130:   Name: GLIBCXX_3.4.9  Flags: none  Version: 13
  0x0140:   Name: GLIBCXX_3.4.11  Flags: none  Version: 11
  0x0150:   Name: CXXABI_1.3  Flags: none  Version: 10
  0x0160:   Name: GLIBCXX_3.4.20  Flags: none  Version: 9
  0x0170:   Name: GLIBCXX_3.4  Flags: none  Version: 8
  0x0180:   Name: GLIBCXX_3.4.21  Flags: none  Version: 6
  0x0190: Version: 1  File: libdl.so.2  Cnt: 1
  0x01a0:   Name: GLIBC_2.2.5  Flags: none  Version: 5
  0x01b0: Version: 1  File: libcudnn.so.7  Cnt: 1
  0x01c0:   Name: libcudnn.so.7  Flags: none  Version: 4
  0x01d0: Version: 1  File: libc.so.6  Cnt: 6
  0x01e0:   Name: GLIBC_2.7  Flags: none  Version: 28
  0x01f0:   Name: GLIBC_2.3.4  Flags: none  Version: 20
  0x0200:   Name: GLIBC_2.4  Flags: none  Version: 19
  0x0210:   Name: GLIBC_2.6  Flags: none  Version: 18
  0x0220:   Name: GLIBC_2.2.5  Flags: none  Version: 7
  0x0230:   Name: GLIBC_2.11  Flags: none  Version: 3

So what either would see if ruy used or other methods to check with?
What if I use a python - which library to check?

anyway - I'll investigate more :)

Saduf2019
added
comp:liteTF Lite related issues
and removed
stat:awaiting responseStatus - Awaiting response from author
on May 27, 2020
Saduf2019
assigned
jvishnuvardhan
and unassigned
Saduf2019
on May 27, 2020
talumbau
self-assigned this
on May 27, 2020
talumbau

talumbau commented on May 27, 2020

@talumbau
talumbau
on May 27, 2020
Member

Hi,

Hopefully I can help you but I just want to make sure I understand the case you are interested in. Please add additional clarification if my answers below don't help. My understanding is that you are concerned with enabling/detecting enablement of Ruy in (at least) one of the following scenarios:

  1. Create a build of TF Lite that has Ruy enabled.

Answer: with a bazel build, add the command line flag --define=tflite_with_ruy=true to the bazel build command. This is already the default for ARM builds, but it does no harm to add it. For Makefile-based builds, ensure that -DTFLITE_WITH_RUY is added to CXXFLAGS, as shown here:

tensorflow/tensorflow/lite/tools/make/Makefile

Line 184 in 5802195

CXXFLAGS += -DTFLITE_WITH_RUY

  1. You have a TFLite binary from somewhere and wish to inspect it to see if Ruy is enabled without executing the binary.
    Answer: there's no supported way to do this, but one could imagine a few ideas.

  2. You have the TF Lite library linked to your application and you wish to, at run time, determine if Ruy is being used for matrix multiplication.
    Answer: there is also no supported way to check here, i.e. a flag of some kind, but you can always compile with -DRUY_PROFILER and then run your program to make sure you get Ruy for the expected GEMM operations. The output at the end of program execution will tell you which GEMMs were done with Ruy (with size info) as well as what Ruy path was selected and how much time was spent on various parts of the GEMM operation.

Does that help?

jvishnuvardhan
removed their assignment
on May 27, 2020
peter197321

peter197321 commented on Jun 1, 2020

@peter197321
peter197321
on Jun 1, 2020
Author

Thank you.
Is it apply for the Cross-compile build of TensorFlow Lite for ARM64 board?

peter197321
closed this as completedon Jun 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

comp:liteTF Lite related issuestype:supportSupport issues

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @talumbau@jvishnuvardhan@peter197321@Saduf2019

      Issue actions
        ruy optimized library enabling for tf-lite · Issue #39812 · tensorflow/tensorflow