Skip to content

Setup Flutter engine environment and build engine on M1 (Apple Silicon ARM64) #96745

New issue
New issue
Closed
Closed
Setup Flutter engine environment and build engine on M1 (Apple Silicon ARM64)#96745
Assignees
cbracken
Labels
P3Issues that are less important to the Flutter projecta: desktopRunning on desktopd: wikiFlutter or engine wikisengineflutter/engine repository. See also e: labels.platform-macBuilding on or for macOS specifically
@eggfly

Description

@eggfly
eggfly
opened on Jan 17, 2022 · edited by ColdPaleLight
Member

Problem 😭

There are some errors if we setup flutter engine environment in standard way, including:

  • gclient sync cannot find third_party/android_tools (including ndk and sdk) and third_party/java packages (openjdk) in arm64 on M1 chip macOS
  • Cannot download prebuilt Dart SDK in arm64, and some header in Dart check 32/64 bit machine matching failed
  • tools/gn shows //out/android_debug/clang_x64/gen_snapshot is not the right target, but it generates a target //out/android_debug/clang_arm64/gen_snapshot instead and failed
  • objcopy command try to find the wrong path
  • Some other errors

Related errors and failure images:
image

See BUILD ON95 oh1ch CRused PNe r1le te Be snecdeg

Pasted Graphic 2

Pasted Graphic 5

Workaround 😊

So I dig into the var host_cpu and find it's maybe originally provided by depot_tools, and I temporarily do some modification, force cpu_arch detech logics to x64 in depot_tools as a workaround:

depot_tools$ git diff 
diff --git a/cipd b/cipd
index 91fbe165..d1903d77 100755
--- a/cipd
+++ b/cipd
@@ -46,13 +46,13 @@ if [ -z $ARCH ]; then
       ARCH="${UNAME}"
       ;;
     aarch64)
-      ARCH=arm64
+      ARCH=amd64
       ;;
     armv7l)
       ARCH=armv6l
       ;;
     arm*)
-      ARCH="${UNAME}"
+      ARCH="amd64"
       ;;
     *86)
       ARCH=386
diff --git a/detect_host_arch.py b/detect_host_arch.py
index cf4bfec7..6a4e8566 100755
--- a/detect_host_arch.py
+++ b/detect_host_arch.py
@@ -44,7 +44,7 @@ def HostArch():
   if host_arch == 'arm64' and platform.architecture()[0] == '32bit':
     host_arch = 'arm'
 
-  return host_arch
+  return 'x64'
 
 def DoMain(_):
   """Hook to be called from gyp without starting a separate python
diff --git a/gclient.py b/gclient.py
index 446be28d..71b087b5 100755
--- a/gclient.py
+++ b/gclient.py
@@ -1283,7 +1283,7 @@ class Dependency(gclient_utils.WorkItem, DependencySettings):
         'checkout_ppc': 'ppc' in self.target_cpu,
         'checkout_s390': 's390' in self.target_cpu,
         'checkout_x64': 'x64' in self.target_cpu,
-        'host_cpu': detect_host_arch.HostArch(),
+        'host_cpu': 'x64',
     }

And also we should let it download in src/flutter/DEPS: ( or maybe this is not a must after cheating in depot_tools):

flutter_engine/src/flutter$ git diff 
diff --git a/DEPS b/DEPS
index c72b4d7e3d..32d7b0a97a 100644
--- a/DEPS
+++ b/DEPS
@@ -66,7 +66,7 @@ vars = {
   'download_dart_sdk': True,
 
   # Checkout Android dependencies only on platforms where we build for Android targets.
-  'download_android_deps': 'host_cpu == "x64" and (host_os == "mac" or host_os == "linux")',
+  'download_android_deps': '(host_os == "mac" or host_os == "linux")',
 
   # Checkout Windows dependencies only if we are building on Windows.
   'download_windows_deps' : 'host_os == "win"',

We also need to disable auto update of depot_tools. depot_tools updates itself automatically when running gclient tool. To disable auto update, set the environment variable DEPOT_TOOLS_UPDATE=0 or run ./update_depot_tools_toggle.py --disable .

Do these mods and rerun gclient sync -D it will prompt an arch change, and download the packages rightly.
The cipd will change from Chrome Infra Package Deployer (cipd 2.6.2 (infra/tools/cipd/mac-arm64@PiaJs-2JM4V3hn1ffZH7wT5QV6z5Rt77aJZaCTRfspYC)) to Chrome Infra Package Deployer (cipd 2.6.2 (infra/tools/cipd/mac-amd64@dsOLt767apUdYzHrNZ7zaQzAOZSEBHZ7ncbfUp_0zfoC))

The cpu arch change prompt:

Detected CIPD client platform change to mac-amd64.
Deleting the existing client to trigger the bootstrap...
Chrome Infra Package Deployer (cipd 2.6.2 (infra/tools/cipd/mac-amd64@dsOLt767apUdYzHrNZ7zaQzAOZSEBHZ7ncbfUp_0zfoC))

Usage:  cipd [command] [arguments]

By forcing it from arm64 to x86, maybe it does two things:

  • The macOS with M1 chip can run x86 program with Rosetta 2 translation, the toolchain, packages, android sdk, ndk, jdk and other binary in x86 will run normally
  • Xcode will build the engine source files into the x86 target arch, this is a magical thing but it really does:
    image

So finally it works for me to build engine from source, on MacBook Air M1 ~ 😂

Pasted Graphic 15

image

Activity

eggfly
added
passed first triage
passed secondary triage
engineflutter/engine repository. See also e: labels.
d: wikiFlutter or engine wikis
on Jan 17, 2022
eggfly
changed the title [-]Setup flutter engine environment and build engine on M1 (AppleSilicon)[/-] [+]Setup flutter engine environment and build engine on M1 (AppleSilicon ARM64)[/+] on Jan 17, 2022
eggfly
mentioned this on Jan 17, 2022
eggfly
changed the title [-]Setup flutter engine environment and build engine on M1 (AppleSilicon ARM64)[/-] [+]Setup Flutter engine environment and build engine on M1 (Apple Silicon ARM64)[/+] on Jan 17, 2022
chinmaygarde
added
P3Issues that are less important to the Flutter project
engineflutter/engine repository. See also e: labels.
and removed
engineflutter/engine repository. See also e: labels.
on Jan 18, 2022
omarzl

omarzl commented on Mar 1, 2022

@omarzl
omarzl
on Mar 1, 2022

Really useful! I had the same issue when compiling the iOS engine in M1
But I only applied the workaround in depot_tools, since there isn't a "download iOS deps" as in Android

Anyway here is a zip with the two patches in case anyone just wants apply them
M1FlutterEnginePatchs.zip

cbracken
mentioned this on Mar 30, 2022
cbracken
added
platform-macBuilding on or for macOS specifically
a: desktopRunning on desktop
on Mar 30, 2022
litang0908

litang0908 commented on Apr 5, 2022

@litang0908
litang0908
on Apr 5, 2022

It worked, thanks !

11 remaining items

cbracken
closed this as completedon May 12, 2022
cbracken
mentioned this on May 12, 2022
yx-mike

yx-mike commented on May 19, 2022

@yx-mike
yx-mike
on May 19, 2022 · Hidden as off-topic
cbracken
added this to Flutter Desktopon May 24, 2022
cbracken
moved this to Done in Flutter Desktopon May 24, 2022
github-actions

github-actions commented on Jun 2, 2022

@github-actions
github-actionsbot
on Jun 2, 2022

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

github-actions
locked as resolved and limited conversation to collaborators on Jun 2, 2022
flar

flar commented on Jul 17, 2022

@flar
flar
on Jul 17, 2022
Contributor

I know that the build is only supported using Rosetta, but the android builds don't work even with Rosetta. Is that a known issue or something that broke recently?

flar

flar commented on Jul 17, 2022

@flar
flar
on Jul 17, 2022
Contributor

(Basically the android tools/ndk are not downloaded at all so trying to gn an android build fails early.)

ColdPaleLight

ColdPaleLight commented on Jul 18, 2022

@ColdPaleLight
ColdPaleLight
on Jul 18, 2022
Member

I know that the build is only supported using Rosetta, but the android builds don't work even with Rosetta. Is that a known issue or something that broke recently?

The following workaround works for me.

flutter_engine/src/flutter$ git diff 
diff --git a/DEPS b/DEPS
index c72b4d7e3d..32d7b0a97a 100644
--- a/DEPS
+++ b/DEPS
@@ -66,7 +66,7 @@ vars = {
   'download_dart_sdk': True,
 
   # Checkout Android dependencies only on platforms where we build for Android targets.
-  'download_android_deps': 'host_cpu == "x64" and (host_os == "mac" or host_os == "linux")',
+  'download_android_deps': '(host_os == "mac" or host_os == "linux")',

c.f.
https://github.com/flutter/engine/blob/b438edfd1afd9a89da3e01403d41225a34a91917/DEPS#L63

flar

flar commented on Jul 18, 2022

@flar
flar
on Jul 18, 2022 · edited by flar
Contributor

I know that the build is only supported using Rosetta, but the android builds don't work even with Rosetta. Is that a known issue or something that broke recently?

The following workaround works for me.

flutter_engine/src/flutter$ git diff 
diff --git a/DEPS b/DEPS
index c72b4d7e3d..32d7b0a97a 100644
--- a/DEPS
+++ b/DEPS
@@ -66,7 +66,7 @@ vars = {
   'download_dart_sdk': True,
 
   # Checkout Android dependencies only on platforms where we build for Android targets.
-  'download_android_deps': 'host_cpu == "x64" and (host_os == "mac" or host_os == "linux")',
+  'download_android_deps': '(host_os == "mac" or host_os == "linux")',

c.f. https://github.com/flutter/engine/blob/b438edfd1afd9a89da3e01403d41225a34a91917/DEPS#L63

I tried that, but I then get errors trying to download various pieces:

Errors:
  failed to resolve perfetto/trace_to_text/mac-arm64@git_tag:v20.1 (line 29): no such tag
  failed to resolve flutter/java/openjdk/mac-arm64@version:11 (line 41): no such package: flutter/java/openjdk/mac-arm64
Error: Command 'cipd ensure -log-level error -root <My Development Dir>/engine -ensure-file /var/folders/66/xlzttnjn66v9f0mgk4m6fy_w00mnw5/T/tmp7ru1kcyo.ensure' returned non-zero exit status 1
Errors:
  failed to resolve perfetto/trace_to_text/mac-arm64@git_tag:v20.1 (line 29): no such tag
  failed to resolve flutter/java/openjdk/mac-arm64@version:11 (line 41): no such package: flutter/java/openjdk/mac-arm64
ColdPaleLight

ColdPaleLight commented on Jul 18, 2022

@ColdPaleLight
ColdPaleLight
on Jul 18, 2022
Member

I know that the build is only supported using Rosetta, but the android builds don't work even with Rosetta. Is that a known issue or something that broke recently?

The following workaround works for me.

flutter_engine/src/flutter$ git diff 
diff --git a/DEPS b/DEPS
index c72b4d7e3d..32d7b0a97a 100644
--- a/DEPS
+++ b/DEPS
@@ -66,7 +66,7 @@ vars = {
   'download_dart_sdk': True,
 
   # Checkout Android dependencies only on platforms where we build for Android targets.
-  'download_android_deps': 'host_cpu == "x64" and (host_os == "mac" or host_os == "linux")',
+  'download_android_deps': '(host_os == "mac" or host_os == "linux")',

c.f. https://github.com/flutter/engine/blob/b438edfd1afd9a89da3e01403d41225a34a91917/DEPS#L63

I tried that, but I then get errors trying to download various pieces:

Errors:
  failed to resolve perfetto/trace_to_text/mac-arm64@git_tag:v20.1 (line 29): no such tag
  failed to resolve flutter/java/openjdk/mac-arm64@version:11 (line 41): no such package: flutter/java/openjdk/mac-arm64
Error: Command 'cipd ensure -log-level error -root <My Development Dir>/engine -ensure-file /var/folders/66/xlzttnjn66v9f0mgk4m6fy_w00mnw5/T/tmp7ru1kcyo.ensure' returned non-zero exit status 1
Errors:
  failed to resolve perfetto/trace_to_text/mac-arm64@git_tag:v20.1 (line 29): no such tag
  failed to resolve flutter/java/openjdk/mac-arm64@version:11 (line 41): no such package: flutter/java/openjdk/mac-arm64

I had the same errors before, when I forgot to change this line of code.
#96745 (comment)

Here is a patch made with all the changes required by depot_tools
depot_tools.patch.zip

In addition to applying the patch of depot_tools, you also need to disable its updates. Good luck :)

# apply patch
git apply ../depot_tools.patch 
git add .
git commit -m"Apple M1 workaround"
# disable depot_tools update
./update_depot_tools_toggle.py --disable
flar

flar commented on Jul 18, 2022

@flar
flar
on Jul 18, 2022
Contributor

I had the same errors before, when I forgot to change this line of code. #96745 (comment)

Here is a patch made with all the changes required by depot_tools depot_tools.patch.zip

In addition to applying the patch of depot_tools, you also need to disable its updates. Good luck :)

So, to answer my question, this doesn't build without workarounds installed in several files which means that the problem still exists (a workaround is not a fix). Perhaps these workarounds will go away if we get the issue to make this work without Rosetta fixed? Either way, this problem still exists.

flar

flar commented on Jul 24, 2022

@flar
flar
on Jul 24, 2022
Contributor

Filed #108241 for the remaining work to enable Android builds on Apple Silicon.

flar

flar commented on Jul 27, 2022

@flar
flar
on Jul 27, 2022
Contributor

#108241 is now closed as fixed.

I verified by creating a brand new engine directory with no modifications to my depot_tools and was able to gsync, use gn to create 3 flavors of android build (debug, profile, release) and build all 3.

We should no longer require any workarounds to build host, iOS, or Android engines on Apple Silicon Macs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P3Issues that are less important to the Flutter projecta: desktopRunning on desktopd: wikiFlutter or engine wikisengineflutter/engine repository. See also e: labels.platform-macBuilding on or for macOS specifically

Type

No type

Projects

Flutter Desktop

Status

Done

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @chinmaygarde@cbracken@eggfly@kismet-@yx-mike

      Issue actions
        Setup Flutter engine environment and build engine on M1 (Apple Silicon ARM64) · Issue #96745 · flutter/flutter