Skip to content

"deno compile" into executable #986

Closed
@matthewmueller

Description

@matthewmueller

I was wondering if there's any plan to support compiling deno scripts into a single executable?

I saw "Single executable" as a feature, but the code that follows seems to suggest it's deno as the single executable, not the programs executed with it.

It'd be awesome to support something like go build main.go or something like zeit/pkg.

Activity

bartlomieju

bartlomieju commented on Feb 16, 2019

@bartlomieju
Member

CC @ry

kitsonk

kitsonk commented on Feb 16, 2019

@kitsonk
Contributor

I believe this is still on the long term roadmap.

ry

ry commented on Feb 18, 2019

@ry
Member

I'm into the idea - and it's quite doable with our infrastructure. I'd estimate ~2 weeks of my time to get it working. It seems very cool, but it's unclear to me what the actual use cases are - do you have any? I also think there's some more important features that should come first.

matthewmueller

matthewmueller commented on Feb 19, 2019

@matthewmueller
Author

The main use case is to simplify deploys – you don't need to care about what's installed on the server because it's all present in that single executable, including the deno executable for the given architecture and OS.

You also tend to find issues at compile time, rather than runtime (e.g. we've all had something something this node module is not found after deploying to production).

kitsonk

kitsonk commented on Feb 19, 2019

@kitsonk
Contributor

You also tend to find issues at compile time, rather than runtime

I don't think a single binary helps or doesn't help in this case. Unless you are going to exercise all the code in tests, you aren't going to find issues. Building it into a binary doesn't help with that.

If you are using TypeScript modules, either local or remote with Deno, and you do a prefetch, all the types will be checked at least. Again, building this into a binary doesn't offer you anything more.

hayd

hayd commented on Feb 19, 2019

@hayd
Contributor

I think the use case is not server deploys but cli/gui apps where you want to lock down the deno version used and it's easier to distribute and call a single file.

The compile before deploy is solved with --prefetch if you include the cached deps.

matthewmueller

matthewmueller commented on Feb 20, 2019

@matthewmueller
Author

I don't think a single binary helps or doesn't help in this case. Unless you are going to exercise all the code in tests, you aren't going to find issues. Building it into a binary doesn't help with that.

I think you're right about this – good point @kitsonk!

I think the use case is not server deploys but cli/gui apps where you want to lock down the deno version used and it's easier to distribute and call a single file.

It depends on what you're doing, but a single binary definitely makes it easier in those cases too!

matthewp

matthewp commented on Feb 28, 2019

@matthewp

Want to reiterate what @hayd says here. I like to create small single-use CLI tools that are essentially "done" and not having to worry about Deno version compatibility would be very enticing. Aside from security patches, it would be nice to have a binary that never needs upgrading.

andyfleming

andyfleming commented on Mar 14, 2019

@andyfleming
Contributor

A couple of use cases for me:

CLI — Having a single executable makes distribution easy. Especially if cross-compiling is easy!

Services built into simple build artifacts — This can be great for small container images, and generally nice for CI/CD workflows.

Support for this would push me over the fence of wanting to use Deno for real projects (once stable). Right now, it's not compelling enough for my use cases.

phil294

phil294 commented on Apr 12, 2019

@phil294

It seems very cool, but it's unclear to me what the actual use cases are

deno could solve an issue that Python and Node have: Ease of running foreign scripts for casual users.

The use case is simple: When my aunt asks me for a programm to organize her holiday pictures on her Windows machine, she does not want to download and install a 30 MB programming environment - she wants a 20 KB do_stuff.exe she can double click.

ry

ry commented on Apr 12, 2019

@ry
Member

Bert and I are actively working on this. Hopefully we’ll have a demo in a two weeks or so.

ry

ry commented on May 30, 2019

@ry
Member

FYI development has stalled on this feature while we are fixing other bugs in the system... but still on my back burner.

109 remaining items

mathiasrw

mathiasrw commented on Dec 8, 2020

@mathiasrw

You did it!!! 1.6.0 is now released including this feature.

deno compile --unstable https://deno.land/std@0.79.0/examples/cat.ts will make you an executable version of the module.

  • Startup time is on pair with running via deno!

  • The size of the 47 MB output file is an area of improvement but we knew that already from this thread.

I'm very very excited.

RedactedProfile

RedactedProfile commented on Dec 8, 2020

@RedactedProfile

Now this is a celebration! I am really excited to try this :)

liudonghua123

liudonghua123 commented on Dec 9, 2020

@liudonghua123

Nice feature, I tried deno compile --unstable https://deno.land/std@0.79.0/examples/cat.ts, and it produced a 31.4 MB sized cat.exe. Look forward to a smaller size in the future.

awkj

awkj commented on Dec 9, 2020

@awkj

excited ! I will use deno instead of shell

samirdjelal

samirdjelal commented on Dec 9, 2020

@samirdjelal

I've found a way to get Deno output smaller by packing Deno executable before the compile.
if you tried to pack the output after compiling Deno will fail to read the content of the code as the compile subcommand will be embed the JS code in the end of the binary as bytecode, it can be seen using any Hex Editor.

So remember:

  1. pack Deno binary.
  2. compile your project files using the packed Deno binary.

image

Deno size reduced from 32MB to 11MB using UPX.

image

[EDIT]

UPX can trigger the anti-virus as false positive malware. we can try another packer/compressor!
Also extra time can be taken to unpack the binary in-memory (~1s for UPX).

MarkTiedemann

MarkTiedemann commented on Dec 9, 2020

@MarkTiedemann
Contributor

Note that packing Deno may make it significantly slower. If packed, executing deno -V takes 1s instead of 20ms on my machine.

@echo off

:: Download upx
curl -LO https://github.com/upx/upx/releases/download/v3.96/upx-3.96-win64.zip
tar xf upx-3.96-win64.zip
move upx-3.96-win64\upx.exe .

:: Download deno
curl -LO https://github.com/denoland/deno/releases/download/v1.6.0/deno-x86_64-pc-windows-msvc.zip
tar xf deno-x86_64-pc-windows-msvc.zip

:: Download timeit
curl -LO https://github.com/MarkTiedemann/rktools2k3/raw/master/timeit.exe

:: Pack deno
upx --ultra-brute -o deno.packed.exe deno.exe

:: Time deno startup (default)
timeit deno.exe -V
:: deno 1.6.0
:: 
:: Version Number:   Windows NT 6.2 (Build 9200)
:: Exit Time:        10:36 pm, Wednesday, December 9 2020
:: Elapsed Time:     0:00:00.020
:: Process Time:     0:00:00.000
:: System Calls:     4496
:: Context Switches: 1444
:: Page Faults:      2303
:: Bytes Read:       2568
:: Bytes Written:    23152
:: Bytes Other:      65540

:: Time deno startup (packed)
timeit deno.packed.exe -V
:: deno 1.6.0
:: 
:: Version Number:   Windows NT 6.2 (Build 9200)
:: Exit Time:        10:36 pm, Wednesday, December 9 2020
:: Elapsed Time:     0:00:00.975
:: Process Time:     0:00:00.968
:: System Calls:     32825
:: Context Switches: 18158
:: Page Faults:      12024
:: Bytes Read:       191782
:: Bytes Written:    554334
:: Bytes Other:      70106

Also, be careful about shipping packed binaries to users: Anti-virus software, including Windows Defender, is more likely to flag your binary and prevent it from running in the first place. Since packing obfuscates the binary to some degree, it is sometimes considered "defense evasion".

samirdjelal

samirdjelal commented on Dec 9, 2020

@samirdjelal

I just mentioned that we can pack it before compiling,
However you are right, UPX will be flagged as false positive by few AVs and it will take extra time to unpack the app in-memory!
and maybe obfuscating the JS before embedding to the end of the binary will be great.
It is a starting point.

coreybutler

coreybutler commented on Dec 9, 2020

@coreybutler

@MarkTiedemann has some valid points, but it's also worth mentioning a few additional points:

Antivirus

To better handle antivirus, code sign your executables using a trusted certificate. Most of the antivirus apps will account for that and allow the app to run.

Boot Time

While there may be a delay, remember to consider the usability of the executable. There are some cases where a 1s delay would be kind of annoying, but there are also many use cases where a 1s delay would go completely unnoticed by end users.

--

Other Considerations

Remember there are other ways to reduce the size of the output, primarily by understanding everything being included in your release. It's easy to "import everything" and not consider where there may be overlap. Perhaps you don't need that hefty logging library when console.log will do. Point: be mindful of what goes in your apps.

If you're interested in how others are using this...

I've been working with a CLI-based dev approach for awhile now, where this feature will come in handy. I reflected on what this means to me in a small piece called JavaScript Executables. So, if anyone is browsing these issues looking for examples or just trying to understand how others are using this feature, here's another resource.

KaKi87

KaKi87 commented on Dec 9, 2020

@KaKi87

sign your executables using a trusted certificate.

Are there free certificate providers for executables as there are for HTTPS ?

coreybutler

coreybutler commented on Dec 9, 2020

@coreybutler

@KaKi87 - nothing free that I'm aware of. It requires a greater degree of trust. An HTTPS cert is hosted, whereas a local executable can (conceivably) do just about anything once installed. That's where things like background checks (which are common for paid certs) are useful. You have to have at least some degree of verifiable legitimacy to get a cert. Unfortunately, that usually means paying. There's a whole thread about why LetsEncrypt does not provide free code signing certs at https://community.letsencrypt.org/t/do-you-support-code-signing/370

KaKi87

KaKi87 commented on Dec 10, 2020

@KaKi87

I think Let's Encrypt could achieve viable code signing from domain names owners using WHOIS.
WHOIS information is directly provided by hosters, which contains either an organization name, an individual name, or both.
That was the problem pointed in the linked topic, right ?

samirdjelal

samirdjelal commented on Dec 10, 2020

@samirdjelal

I don't think you can find a free code-signing, as it is all about trust factor, if you can get free cert it means malware authors can get it too and use it to bypass AV, however there is open source code signing code for €25!
https://shop.certum.eu/open-source-code-signing-code.html

daotoad

daotoad commented on Dec 10, 2020

@daotoad

Years ago, I used PAR::Packer and ActiveState's now defunct PerlApp to create distributable perl code. Both tools used a strategy of packing the files, then caching the unpacked files after the first run. The end result was a significant speed boost on subsequent executions. This strategy works very nicely.

I don't have any good ideas on the antivirus flagging of packed files. It was an occasional issue with PerlApp and is part of the reason ActiveState gave up on it.

kitsonk

kitsonk commented on Dec 10, 2020

@kitsonk
Contributor

Enhancements to the feature should probably be discussed as a seperate thread in Discussions instead of comment on a closed issue where the thoughts and feedback will likely be lost or ignored.

locked as resolved and limited conversation to collaborators on Dec 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    clirelated to cli/ dirfeatnew feature (which has been agreed to/accepted)

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @ry@aral@rivy@srikumarks@matthewmueller

      Issue actions

        "deno compile" into executable · Issue #986 · denoland/deno