... | @@ -2,19 +2,19 @@ |
... | @@ -2,19 +2,19 @@ |
|
|
|
|
|
**Damned Lies** is a Python Web application based on [Django](https://www.djangoproject.com/) to manage data, templates and the user interaction. It is used to coordinate the translations done by contributors distributed in language teams. It is connected to our software forge, [gitlab.gnome.org](https://gitlab.gnome.org) to send the translated [`.po`](https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html) files to the associated projects.
|
|
**Damned Lies** is a Python Web application based on [Django](https://www.djangoproject.com/) to manage data, templates and the user interaction. It is used to coordinate the translations done by contributors distributed in language teams. It is connected to our software forge, [gitlab.gnome.org](https://gitlab.gnome.org) to send the translated [`.po`](https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html) files to the associated projects.
|
|
|
|
|
|
For the CSS, it uses a custom theme based on [Boostrap](https://getbootstrap.com/) (Currently version 5.X [30/11/2022]). This theme is called [`gnome-bootstrap-theme`](https://gitlab.gnome.org/Infrastructure/gnome-bootstrap-theme) and also hosted on [gitlab.gnome.org](https://gitlab.gnome.org).
|
|
For the CSS, it uses a custom theme based on [Boostrap](https://getbootstrap.com/) (Currently version 5.X [10/11/2023]). This theme is called [`gnome-bootstrap-theme`](https://gitlab.gnome.org/Infrastructure/gnome-bootstrap-theme) and also hosted on [gitlab.gnome.org](https://gitlab.gnome.org).
|
|
|
|
|
|
It benefits from multiple system tools provided by the [`gettext`](https://www.gnu.org/software/gettext/) project and [`git`](https://git-scm.com/), in order to send translations to the forge.
|
|
It benefits from multiple system tools provided by the [`gettext`](https://www.gnu.org/software/gettext/) project and [`git`](https://git-scm.com/), in order to send translations to the forge.
|
|
|
|
|
|
## Environment
|
|
## Environment
|
|
|
|
|
|
**Damned Lies** is deployed in a Kubernetes environment hosted by GNOME/Red Hat that relies on the [OpenShift technology](https://www.redhat.com/en/technologies/cloud-computing/openshift).
|
|
**Damned Lies** is deployed in a Kubernetes environment hosted by GNOME that relies on the [OpenShift technology](https://www.redhat.com/en/technologies/cloud-computing/openshift).
|
|
|
|
|
|
As a consequence, the application is deployed in containers that follow the OCI principles and are compatible with `podman` and `buildah` (`docker` as well). The files to build the container image are located in the `containers` directory. Images are built with `buildah`.
|
|
As a consequence, the application is deployed in containers that follow the OCI principles and are compatible with `podman` and `buildah`. The files to build the container image are located in the `containers` directory. Images are built with `buildah`.
|
|
|
|
|
|
It’s a two-step building. First, a runtime is built with all the necessary dependencies to run the application. The runtime is based on [Fedora OCI images](https://hub.docker.com/_/fedora), the target being the old stable version. This runtime is used both in the Gitlab CI progress and to deploy the application. Then, to create the production images (*testing*, *staging* and *production*), the build process is based on these runtimes.
|
|
It’s a two-step building. First, a runtime is built with all the necessary dependencies to run the application. The runtime is based on [`fedora-minimal` OCI images](https://quay.io/repository/fedora/fedora-minimal). We always use the `latest` tag to remain up-to-date with the last security updates. This runtime is used both in the Gitlab CI process and to deploy the application. Then, to create the production images (`testing`, `staging` and `production`), the build process is based on these runtimes.
|
|
|
|
|
|
**Note**: the user ID in the `buildah` files has a specific UID/GID (`1001010000`) which is managed by OpenShift in order to run in an unprivileged environment. See the [documentation](https://cloud.redhat.com/blog/a-guide-to-openshift-and-uids) for more information. This user (the one that runs the process) is called `l10n` inside the container.
|
|
**Note**: the user ID in the `buildah` files has a specific UID (`1001010000`) which is managed by OpenShift in order to run in an unprivileged environment. See the [documentation](https://cloud.redhat.com/blog/a-guide-to-openshift-and-uids) for more information. This UID depends on the OpenShift configuration you can find in the project YAML files.
|
|
|
|
|
|
## Image configuration
|
|
## Image configuration
|
|
|
|
|
... | @@ -22,7 +22,7 @@ The **Damned Lies** configuration is based on a few files. They are Jinja2 templ |
... | @@ -22,7 +22,7 @@ The **Damned Lies** configuration is based on a few files. They are Jinja2 templ |
|
|
|
|
|
1. `entrypoint.sh.jinja2`: the image entrypoint, a shell script that runs the Django commands to prepare the application and that replaces (see the `sed` commands) temporary variables in the `local_settings.py` files with ones found in the deployment environment. All those variable are set in the OpenShift environment in order to pass each pod sensitive data or on the fly configuration.
|
|
1. `entrypoint.sh.jinja2`: the image entrypoint, a shell script that runs the Django commands to prepare the application and that replaces (see the `sed` commands) temporary variables in the `local_settings.py` files with ones found in the deployment environment. All those variable are set in the OpenShift environment in order to pass each pod sensitive data or on the fly configuration.
|
|
1. `damnedlies/local_settings.py` in the **Damned Lies** repository. This file is used to configure the application itself. The template of this file is located in `containers/production/configuration/local_settings.py.jinja2`. In this file, all the configurations that **must not** be visible from outside are prefixed with the `$` sign.
|
|
1. `damnedlies/local_settings.py` in the **Damned Lies** repository. This file is used to configure the application itself. The template of this file is located in `containers/production/configuration/local_settings.py.jinja2`. In this file, all the configurations that **must not** be visible from outside are prefixed with the `$` sign.
|
|
1. `httpd.conf.jinja2`: The Apache HTTPd configuration file. A web server runs in the container to serve the Web application with [`WSGI`](https://wsgi.readthedocs.io/en/latest/what.html), which is the [recommended method to deploy a Django application](https://docs.djangoproject.com/en/4.1/howto/deployment/).
|
|
1. `httpd.conf.jinja2`: The Apache HTTPd configuration file. A web server runs in the container to serve the Web application with [`WSGI`](https://wsgi.readthedocs.io/en/latest/what.html), which is the [recommended method to deploy a Django application](https://docs.djangoproject.com/en/4.2/howto/deployment/).
|
|
|
|
|
|
## Container repository
|
|
## Container repository
|
|
|
|
|
... | @@ -44,18 +44,18 @@ OCI images for **staging** and **production** are strictly the same ones and bas |
... | @@ -44,18 +44,18 @@ OCI images for **staging** and **production** are strictly the same ones and bas |
|
### Filesystem access
|
|
### Filesystem access
|
|
|
|
|
|
* Files distributed by the **Damned Lies** application are located in a folder set in the Django `settings.py` file (`DATADIR`). This directory mush be shared through OpenShift to offer persistent storage (PVC - Persistent Volume Claim in OpenShift).
|
|
* Files distributed by the **Damned Lies** application are located in a folder set in the Django `settings.py` file (`DATADIR`). This directory mush be shared through OpenShift to offer persistent storage (PVC - Persistent Volume Claim in OpenShift).
|
|
* A `/home/l10n/.ssh` file is necessary. If contains the SSH configuration to connect to the [gitlab.gnome.org](gitlab.gnome.org) application and send translations. (Secret Volume in OpenShift). It contains two files: `config` (`man 5 ssh_config`) and a private key (referenced in the `config` file). For instance:
|
|
* A `/.ssh` file is necessary. If contains the SSH configuration to connect to the [gitlab.gnome.org](gitlab.gnome.org) application and send translations. (Secret Volume in OpenShift). It contains two files: `config` (`man 5 ssh_config`) and a private key (referenced in the `config` file). For instance:
|
|
```text
|
|
```text
|
|
Host gitlab.gnome.org
|
|
Host gitlab.gnome.org
|
|
Hostname ssh.gitlab.gnome.org
|
|
Hostname ssh.gitlab.gnome.org
|
|
User git
|
|
User git
|
|
IdentityFile ~/.ssh/ssh-privatekey
|
|
IdentityFile /.ssh/ssh-privatekey
|
|
StrictHostKeyChecking no
|
|
StrictHostKeyChecking no
|
|
UserKnownHostsFile=/dev/null
|
|
UserKnownHostsFile=/dev/null
|
|
```
|
|
```
|
|
So the private key will be called `ssh-privatekey`.
|
|
So the private key will be called `ssh-privatekey`.
|
|
|
|
|
|
* A `home/l10n/.gitconfig` file is expected, in order to declare the identity of the commiter (the application itself). It could be, for instance:
|
|
* A `/.gitconfig` file is expected, in order to declare the identity of the commiter (the application itself). It could be, for instance:
|
|
```text
|
|
```text
|
|
[user]
|
|
[user]
|
|
email = gnome-sysadmin@gnome.org
|
|
email = gnome-sysadmin@gnome.org
|
... | @@ -64,12 +64,16 @@ So the private key will be called `ssh-privatekey`. |
... | @@ -64,12 +64,16 @@ So the private key will be called `ssh-privatekey`. |
|
|
|
|
|
### Cron jobs for maintenance tasks
|
|
### Cron jobs for maintenance tasks
|
|
|
|
|
|
There are two jobs that are done regularly in order to maintain consistency in the Damned Lies data.
|
|
There are two jobs that are done regularly in order to maintain consistency in the Damned Lies data. These cron jobs are also created daily (or more frequently) by OpenShift.
|
|
|
|
|
|
1. `manage.py run-maintenance` (every day). As described in `stats/management/run-maintenance.py`, it cleans not activated accounts and obsolete ones, cleans old actions and removes tar files.
|
|
1. `manage.py run-maintenance` (every day). As described in `stats/management/run-maintenance.py`, it cleans not activated accounts and obsolete ones, cleans old actions and removes tar files.
|
|
|
|
|
|
1. `manage.py update-stats --non-gnome` (twice a day). Regenerates the statistics for non GNOME projects (externally hosted).
|
|
1. `manage.py update-stats --non-gnome` (twice a day). Regenerates the statistics for non GNOME projects (externally hosted).
|
|
|
|
|
|
|
|
It is possible to run manually these jobs from the command line by calling, for instance, the `oc` command:
|
|
|
|
```bash
|
|
|
|
oc create job --from=cronjob/damned-lies-run-maintenance cron-job-maintenance
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
|
... | | ... | |