|
|
# RFC: Filesystem permissions and environments with Docker
|
|
|
|
|
|
|
|
|
|
|
|
In its current implementation, Drupalizer has a hard dependency on Docker to manage the configuration of development environment, which currently includes: PHP, Apache, MySQL, NPM and Python.
|
|
|
|
|
|
|
|
|
|
|
|
By using Linux namespaces, Docker creates an independent view of system resources for every container. A command run in a container, does not inherit environment or attributes of the shell where it was invoked.
|
|
|
|
|
|
|
|
|
|
|
|
This way, if a script writes a file to disk, it will be created with different UIDs when the script is executed inside and outside of Docker.
|
|
|
|
|
|
|
|
|
|
|
|
We will try to execute `fab init` in a clear project that uses Drupalizer and examine the properties of all files that were created. The `init` task will be executed for the first time, so the `src/drupal/` directory does not contain anything yet.
|
|
|
|
|
|
|
|
|
```
|
|
|
$ fab init
|
|
|
[...]
|
|
|
|
|
|
$ find src/drupal/ -type f -exec stat -c %U {} \; | sort -u
|
|
|
root
|
|
|
ubuntu
|
|
|
www-data
|
|
|
```
|
|
|
|
|
|
|
|
|
The execution of `init` task produced files owned by three different system users, among which is the `root` user who ends up owning at least the following file `src/drupal/sites/all/drush/aliases.drushrc.php`, and the source of all modules and themes that were downloaded as dependencies during site installation.
|
|
|
|
|
|
|
|
|
|
|
|
It blocks automation (at least) in the following ways:
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
1) It is impossible to run two successful sequential builds in Jenkins without manual intervention.
|
|
|
|
|
|
|
|
|
|
|
|
During a Jenkins build, files in the build workspace will be removed in two cases: (a) automatically by Jenkins at the beginning of the build (which is what Jenkins 2 does); (b) by Drush `make` command which will try to move Drupal source tree into `src/drupal/` after downloading it. In both cases, continuous deployment with Jenkins will be blocked until someone with root access manually deletes the whole build workspace.
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
2) It is impossible to run `fab init && fab release && fab deploy` as a single command in neither local environment, nor Jenkins. As its first step, Drupalizer `deploy` command will try to empty the contents of the `src/` directory and will instantly fail because it cannot remove files not owned by the current user.
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
3) If we choose, for some project, to commit compiled CSS files alongside with the SASS sources, it will break `git pull` command in the following scenario:
|
|
|
|
|
|
|
|
|
|
|
|
a) execute CSS preprocessor build task from inside of Docker environment;
|
|
|
b) grunt/gulp deletes and recreates all CSS files but now they are owned by the `root` user;
|
|
|
c) execute `git pull` to fetch new commits that contain modifications of those CSS files;
|
|
|
d) Git will not be able to apply changes to files not owned by the current user and fill exit with an error.
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
4) Translation sources, cache files and images written to disk by Apache and Drush from inside of Docker cannot be removed or updated in local environment without `sudo` which requires manual intervention. This may happen if Drush is used both inside and outside of Docker for debugging purposes, for example.
|
|
|
|
|
|
|
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
|
|
## Mitigation
|
|
|
|
|
|
|
|
|
|
|
|
In case of files owned by `www-data`, it is possible to mitigate the problem of ownership by adding the main user to the `www-data` group. This way, files and directories created by Apache user inside of Docker can be deleted in local environment. But this approach has the following drawbacks:
|
|
|
|
|
|
|
|
|
|
|
|
a) the main user (ubuntu, jenkins, etc.) has to be manually added to `www-data` group on every host;
|
|
|
|
|
|
|
|
|
|
|
|
b) all files and directories created by Apache inside of Docker must be group-writable which is *not* the default creation mode. This, again, requires manual configuration of `umask` on each and every created container (this breaks continuous deployment), or this has to be specified in Dockerfile;
|
|
|
|
|
|
|
|
|
|
|
|
c) this approach is not portable because the name and UID of Apache user is not guaranteed to be the same on all Linux machines. For example, Ubuntu uses `www-data` user with UID 33, while CentOS uses `apache` user with UID 48. So if Drupalizer is used on CentOS host, file operations will still require `sudo` command.
|
|
|
|
|
|
|
|
|
|
|
|
d) this approach cannot be scaled to work with files owned by `root` user. Only the `root` user can remove those files and directories.
|
|
|
|
|
|
|
|
|
|
|
|
## Proposed solution
|
|
|
|
|
|
|
|
|
|
|
|
In order to turn Drupalizer into a reliable continuous deployment tool that will not unpredictably block builds, it must be to ensured that all executed tasks and commands always *run with same UID and GID* of the main user who invokes the `fab` command.
|
|
|
|
|
|
|
|
|
|
|
|
This is a more predictable behavior that will contribute to transparency of environments and decrease dependency on Docker. For example, if file ownership is properly maintained, it does not matter if `drush dl features` is executed in local environment or inside of Docker - the result will be the same.
|
|
|
|
|
|
|
|
|
|
|
|
To achieve this behavior, it is necessary to implement the following steps:
|
|
|
|
|
|
|
|
|
|
|
|
1. Detect the UID and GID when any Fabric task is executed or Docker image is built.
|
|
|
|
|
|
2. Configure Apache to run with these UID and GID.
|
|
|
|
|
|
3. Connect to the Docker container using the correct user in order to have these UID and GID after login; OR use `sudo` to run every command as the correct user.
|
|
|
|
|
|
4. Ensure that none of the commands executed by Fabric tasks require root privilege on the container in order to simplify execution of commands. All configuration of the Docker environment must be done either when an image is built or when a container is first created.
|
|
|
|
|
|
|
|
|
|
|
|
## Drawbacks
|
|
|
|
|
|
|
|
|
|
|
|
While this solution makes Drupalizer usage more predictable, it introduces certain new expectations on how the Docker image will behave and interact with Drupalizer. As a minimum, Docker image should be able to receive UID and GID that we want to use to execute commands.
|
|
|
|
|
|
|
|
|
|
|
|
The most simple approach would be to always use a pre-configured `Dockerfile` included into Drupalizer distribution. But this will make Drupalizer less flexible because projects that rely on it will loose ability to use custom Docker images.
|
|
|
|
|
|
|
|
|
|
|
|
An alternative approach will be to ship Drupalizer with a set of integration scripts, and simply require those scripts to be included into `Dockerfile`. |
|
|
\ No newline at end of file |