In short, the cause of all these drama is the union filesystem (read here). If you work on an image and have performed multiple commits (either through DockerFile or manually), then you might run into this problem later in the stage. The large files might have been committed in one of the layers during the earlier stage.
Eg.
(1) We have this image called "centos63:omnibus731fp8-hm2000" which is about 2.11GB in size.
(2) Next, let us run the image and delete a very large directory in there.
 |
(2) Next, let us run the image and delete a very large directory in there.
 |
(3) To persist the change, I committed the image and tagged the new image as "centos63:omnibus731fp8-hm2000-rm".
 |
(4) Surprise, surprise! The size is still 2.11GB!?!?
 |
(5) What happened? Let's check it out. Is the large directory still there? Nope!!!
 |
(6) Then why didn't the "rm" work?
You can see that the large directory was most probably (sorry I can't tell for sure because the image was created with a previous very bad habit of manual commits :) added in image "28729cfb27de" that was created very early in the stage. Ever since, there were multiple commits (represented by those multiple different images in the history).
Another thing to pay attention to is the "rm" command that was persisted in image "ae11f957b955" (latest - highlighted in BLUE box) and the size was only 7 bytes!!! It shows clearly that Docker has only recorded the "rm" command and not the result of it.
That is the "power" of an union filesystem. But sorry, your large directory was still technically in there.
You can see that the large directory was most probably (sorry I can't tell for sure because the image was created with a previous very bad habit of manual commits :) added in image "28729cfb27de" that was created very early in the stage. Ever since, there were multiple commits (represented by those multiple different images in the history).
Another thing to pay attention to is the "rm" command that was persisted in image "ae11f957b955" (latest - highlighted in BLUE box) and the size was only 7 bytes!!! It shows clearly that Docker has only recorded the "rm" command and not the result of it.
That is the "power" of an union filesystem. But sorry, your large directory was still technically in there.
 |
(7) So, what now? How do you trim those extra fat off? The answer is with this awesome tool called docker-squash!
I executed the tool with an option to name the output image "centos63:omnibus731fp8-hm2000-squashed".
You can see how the tool successfully determined all the layers belonging to the image (check the UUID).
 |
(8) From the output of the tool, it seems that it has successfully trimmed down the image.
 |
(9) Let's verify whether the new image "omnibus731fp8-hm2000-squashed" is significantly smaller in size.
It seems that the tool does deliver after all! :)
 |
I have read that Docker Inc is working on having this feature (shrinking the image size) built-in. So, while waiting for it, you have this tool!
No comments:
Post a Comment