Skip to content

dockerfile: Split the ADD command #3050

Closed
@amluto

Description

@amluto

Currently the ADD command is IMO far too magical. It can add local and remote files. It will sometimes untar a file and it will sometimes not untar a file. If a file is a tarball that you want to copy, you accidentally untar it. If the file is a tarball in some unrecognized compressed format that you want to untar, you accidentally copy it.

There's also essentially no security for downloads.

Can we have unambiguous options, please. For example:

COPY-IN <local file or directory> <target in image>

UNTAR <local tarball> <target directory in image>

DOWNLOAD NOVERIFY <URL> <target>

DOWNLOAD SHA256 <sha256 hash> <URL> <target>

etc.

Activity

vieux

vieux commented on Dec 4, 2013

@vieux
Contributor

👍 👍 👍 👍 👍

I agree with you Currently the ADD command is IMO far too magical

Here are my comments:

  • We should keep options in one keyword
  • Dockerfile syntax should be easy to learn, no need to have so many commands
  • Right now with your example there is no way to untar a remote tarball, I would suggest something like:
    • COPY/INJECT -> copy local/remote files/directory/archive in the image
    • UNTAR -> copy local/remote archive in the image and untar them
tianon

tianon commented on Dec 4, 2013

@tianon
Member

Massive +1s, and I love @vieux's suggestion of just COPY and UNTAR commands to remove some of the magic here. This is a major roadblock for a couple of my projects right now.

As a side note, the hash verification stuff was already covered in #2579, and I think that'd be the appropriate place to discuss that further, especially since splitting ADD alone is a big enough topic for one issue. :)

I think the "remote download" items could also do with Last-Modified and/or ETag HEAD lookups for when we eventually get "ADD caching", but that's not really the discussion here either. 👍

amluto

amluto commented on Dec 4, 2013

@amluto
Author

I can work with COPY and UNTAR.

scalp42

scalp42 commented on Dec 8, 2013

@scalp42

explicit > implicit +1

SvenDowideit

SvenDowideit commented on Dec 11, 2013

@SvenDowideit
Contributor

why UNTAR rather than RUN tar zxvf?

ie, INSERT a_tarfile then (do i need to set a WORKDIR?) RUN tar zxvf a_tarfile

first up, INSERT is thus the same as docker insert and when i need to run something...

shykes

shykes commented on Dec 11, 2013

@shykes
Contributor

One difference is that "run tar" depends on the version of tar available in the image. It's better to guarantee the same version everywhere (ideally docker would embed its own implementation) , and insome scenarios the images will not have tar at all.

On Tue, Dec 10, 2013 at 11:05 PM, Sven Dowideit notifications@github.com
wrote:

why UNTAR rather than RUN tar zxvf?
ie, INSERT a_tarfile then (do i need to set a WORKDIR?) RUN tar zxvf a_tarfile

first up, INSERT is thus the same as docker insert and when i need to run something...

Reply to this email directly or view it on GitHub:
#3050 (comment)

SvenDowideit

SvenDowideit commented on Dec 11, 2013

@SvenDowideit
Contributor

understood

tianon

tianon commented on Dec 11, 2013

@tianon
Member

Also because that would double the size of our base images, not to mention where does the original tar come from when we start with nothing?

I think it's very important that we keep the ability to create a layer entirely from a tar (for our base images at least), I would just be extremely grateful for a way to do so with a remote tar too, and if we could disentangle ADD while we're at it, I see that as a win, personally.

tonyarkles

tonyarkles commented on Mar 6, 2014

@tonyarkles

Another vote for disentangling it. I've just run into a problem, and even my hacky workaround wasn't enough to get around the ADD magic.

I've got a tarball in my context (local) that needs to get copied into the image compressed (it gets used later on directly, not decompressed). I started out trying to ADD it, but discovered that it got untarred. As a workaround, I renamed it to .foo instead of .tar.gz and it still got decompressed!

Not sure what my next attempt is going to be, but it'd be awesome if there were a straightforward way to just say "copy this file as-is".

Edit: A final workaround that actually worked:

Before hand:
openssl enc -aes-256-cbc -k password -in myfilename.tar.gz -out myfilename.encrypted

In Dockerfile:
ADD ./myfilename.encrypted /root/
RUN openssl enc -d -aes-256-cbc -k password -in /root/myfilename.encrypted -out /root/myfilename.tar.gz

pda

pda commented on Mar 17, 2014

@pda
Contributor

Dockerfile / docker build should implement minimal functionality, and instead delegate to commands which are available on the CLI as docker <cmd> ….

I think ADD is the biggest offender on this front. Some symptoms:

  • ADD in a Dockerfile is confusing and doesn't have its own documentation page.
  • docker add doesn't exist; no CLI mechanism to recursively add a local directory to an image.

I imagine ADD is tricky because it depends on the build context (tarred $PWD sent to the docker daemon). But I'm sure we can work around that. If running with a build context, use that; if not, tar and send the specified directory to be added to the image.

Maybe this warrants an issue, not a comment, but it seems very relevant here.

pnasrat

pnasrat commented on Apr 19, 2014

@pnasrat
Contributor

We have a bunch of bugs on ADD at the moment. What's the best way to pitch a strawman implementation?

34 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/builderexp/beginnerkind/enhancementEnhancements are not bugs or new features but can improve usability or performance.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @pnasrat@pda@SvenDowideit@shykes@dreamcat4

        Issue actions

          dockerfile: Split the ADD command · Issue #3050 · moby/moby