Skip to content

[vm/ffi] int, long, size_t, uintptr_t, wchar_t, etc #36140

@dcharkes

Description

@dcharkes
Contributor

Update 2022-01-20: This will be available in dart:ffi from Dart 2.17 (and from now on tip of tree). In the mean time this is also available in package:ffi for Dart 2.16 (dev release).

Update 2021-01-07: We will implement this by implementing #42563 and probably adding the common types to package:ffi.

Currently dart:ffi only supports:

  • int8_t
  • int16_t
  • int32_t
  • int64_t
  • uint8_t
  • uint16_t
  • uint32_t
  • uint64_t
  • intptr_t
  • float
  • double

However, many C APIs use C types such as int, long, size_t, uintptr_t, and wchar_t.

We could support these types. The question is if we support these types, then what other types should we support.

An alternative could be to not support these types but provide a way for users to specify types which differ in size per platform.

@jonasfj @sjindel-google

Activity

added
area-core-librarySDK core library issues (core, async, ...); use area-vm or area-web for platform specific libraries.
on Mar 7, 2019
dcharkes

dcharkes commented on Jan 21, 2020

@dcharkes
ContributorAuthor

sizeof(int) on iOS arm64 returns 4. Thus, we cannot safely substitute int with intptr_t.

added a commit that references this issue on Jan 21, 2020
artob

artob commented on May 15, 2020

@artob

From a user perspective, the lack of these standard variable-sized types is probably the single largest remaining deficiency in Dart's FFI support. It does rather complicate making FFI bindings for Dart.

As someone routinely working on a number of FFI bindings for a multitude of different languages (most recently, OpenXR bindings), the specific list of missing, absolutely essential variable-sized types in Dart is:

  • int and unsigned int
  • long and unsigned long
  • size_t (stddef.h)

And some other, somewhat rarer variable-sized types to consider supporting would be:

  • off_t (sys/types.h)
  • ssize_t (sys/types.h)
  • wchar_t (stddef.h)

All the aforementioned types listed in priority order, in terms of how badly and frequently I have needed them.

What's the plan for supporting at least int, long, and size_t?

dcharkes

dcharkes commented on May 20, 2020

@dcharkes
ContributorAuthor

These types are not part of the ABI, and the C standard has a very flexible specification (see this quick explanation). So, a compiler can choose what size/alignment these types have.

dart:ffi needs to be able to know what the size/alignment of types is of the compiled shared object loaded into Dart. The DartVM knows what OS (Linux,Windows,MacOS,iOS,Android) and hardware (x64,ia32,arm32,arm64) it is running on, so it can rely on the ABI and use the types specified in the ABI.

However, dart:ffi does not know with what compiler (and what compiler options) the shared object that is being loaded is compiled. So it can not know how to treat these C types. (Adding a possibly endless list of compilers/compiler options to DartVM is probably not a good approach, because it would require us to change the DartVM every time someone tries to use a new combination.)

It might be the case that for some of these types, all the compilers targeting the various ABIs (OS/Hardware combinations) agree on a size/alignment. Then we could incorporate these types into the ABI-logic. The downside is that if ever some new compiler comes up that does not respect the status quo, stuff breaks.

Does anyone have experience with whether the most common compilers agree on these types are handled in specific ABIs? Do we have any prior art in FFIs in other languages?

Another direction we could go to is to pass in the specs for these types into a bindings generator. The downside here is that one needs to re-run the bindings generator for every compiler/compiler options/hardware/OS combination when releasing an app.

@artob, what are you currently doing to work around this limitation?

artob

artob commented on May 21, 2020

@artob

dart:ffi needs to be able to know what the size/alignment of types is of the compiled shared object loaded into Dart. The DartVM knows what OS (Linux,Windows,MacOS,iOS,Android) and hardware (x64,ia32,arm32,arm64) it is running on, so it can rely on the ABI and use the types specified in the ABI.

However, dart:ffi does not know with what compiler (and what compiler options) the shared object that is being loaded is compiled. So it can not know how to treat these C types. (Adding a possibly endless list of compilers/compiler options to DartVM is probably not a good approach, because it would require us to change the DartVM every time someone tries to use a new combination.)

@dcharkes Respectfully, you might be overthinking this. Just about any FFI for any programming language has the same theoretical problem, yet they all provide these FFI types.

It might be the case that for some of these types, all the compilers targeting the various ABIs (OS/Hardware combinations) agree on a size/alignment. Then we could incorporate these types into the ABI-logic. The downside is that if ever some new compiler comes up that does not respect the status quo, stuff breaks.

Does anyone have experience with whether the most common compilers agree on these types are handled in specific ABIs? Do we have any prior art in FFIs in other languages?

Every single FFI in every single programming language I've worked with--with the sole exception of Dart--supports these basic types: at the very least, int, long, and size_t are universal. Off the top of my head, here's a partial list of FFIs I have prior experience with, and how they map the types I mentioned:

I omitted Go, LuaJIT, Nim, PHP, and Zig from the list, since their FFI implementations all parse C header syntax directly (and implicitly support these types).

@artob, what are you currently doing to work around this limitation?

I'm not. This, plus #35763, is blocking my efforts for nontrivial FFI bindings for Dart and Flutter. It would be a royal pain in the arse to work around both of these blockers, so I'm focusing my efforts elsewhere for now and hoping to come back to all this after a future Dart release announcement.

artob

artob commented on May 21, 2020

@artob

@artob, what are you currently doing to work around this limitation?

Oh, and I might as well mention this here as an aside given that I haven't found an existing issue for it: the lack of general type aliases--for example, defining CLong as a typedef for either Int32 or Int64 at compile time--also hampers creating FFI bindings.

dcharkes

dcharkes commented on May 21, 2020

@dcharkes
ContributorAuthor

Oh, and I might as well mention this here as an aside given that I haven't found an existing issue for it: the lack of general type aliases--for example, defining CLong as a typedef for either Int32 or Int64 at compile time--also hampers creating FFI bindings.

See dart-lang/language#65 for that.

19 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-core-librarySDK core library issues (core, async, ...); use area-vm or area-web for platform specific libraries.library-ffitype-design

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @artob@ndusart@dcharkes@lrhn@alexmercerind

        Issue actions

          [vm/ffi] int, long, size_t, uintptr_t, wchar_t, etc · Issue #36140 · dart-lang/sdk