1. 02 Feb, 2023 2 commits
  2. 29 Jan, 2023 6 commits
  3. 28 Jan, 2023 2 commits
  4. 23 Jan, 2023 2 commits
  5. 16 Jan, 2023 1 commit
  6. 26 Dec, 2022 2 commits
  7. 11 Dec, 2022 4 commits
    • Jehan's avatar
      tools: some more improvements to gimp-check-mirrors.py. · acc697bb
      Jehan authored
      - These "Connection reset by peer" keep happening, and this time, I had one case
        where the final exception was not requests.exceptions.ChunkedEncodingError but
        urllib3.exceptions.ProtocolError. So let's also catch this one. I'm also
        adding ConnectionResetError though it doesn't like this is  ever the last
        exception so far.
      - Remove a trailing closing parenthese added mistakenly with the orange escape
        code.
      - Double the timeout to hopefully not have any more dropped download check.
      acc697bb
    • Jehan's avatar
      tools: add color to the output. · 01590ae8
      Jehan authored
      Gitlab CI can show colored output, so let's add some to more easily find errors
      when looking at the CI (as well as on local tests, of course).
      01590ae8
    • Jehan's avatar
      tools, gitlab-ci: store failed checksum files as artifacts. · bb402c5d
      Jehan authored
      Instead of computing checksum from a file stream during download, first download
      the file on the file system, then compute its checksum and finally delete it
      only if check is OK. The failed files are left as CI artifacts.
      
      This will allow us to manually verify failed files, otherwise the check has no
      meaning. We need to make sure if a failed check is due to download corruption
      (typically I saw that some files in "checksum KO" result were apparently
      bit-to-bit equal (for what was present), yet unfinished content; somehow the
      download had failed even though we had no connection errors) and not malicious
      changes. Basically if we happen to have a rogue mirror, we absolutely need to
      make sure of it by having an actual file to compare and test.
      bb402c5d
    • Jehan's avatar
      tools: better error reporting for "Connection reset by peer" issues. · bf384c46
      Jehan authored
      There was a bunch of failed downloads which were actually "Connection reset by
      peer" but my script was reporting them as None. This should fix it. Note that it
      doesn't look like requests.exceptions.ChunkedEncodingError is really the
      sensible exception which should have been returned. Maybe I should verify if it
      should be reported to the requests module maintainers.
      Also the .strerror property contains None, so I use the first args which
      contains a relevant string instead.
      bf384c46
  8. 10 Dec, 2022 2 commits
    • Jehan's avatar
      tools: better handle 404 errors in gimp-check-mirrors.py. · 5c8ffa8d
      Jehan authored
      - gimputils.version.find_latest() now also returns the release date specific for
        a tarball or last package revision release.
      - In gimp-check-mirrors.py: display the release date of checked files (when they
        are taken from the json file with find_latest()). This allows to get a better
        idea of whether missing files are within acceptable delay for a mirror giving
        404 response. E.g. if a mirror still miss some files weeks after release, it
        might be a problem.
      - No need to compute a checksum and display "checksum KO" for HTTP response
        other than 200 (success). It doesn't make sense to say the checksum was in
        error when we could not get the file at all (then our script still count it as
        error, but it's not a corrupted file error).
      5c8ffa8d
    • Jehan's avatar
      tools: fix termination of gimp-check-mirrors.py script. · 9da615f1
      Jehan authored
      Fixing:
      
      > Traceback (most recent call last):
      >   File "/builds/Infrastructure/gimp-web/tools/downloads/gimp-check-mirrors.py", line 161, in <module>
      >     if origin_executor is not None:
      > NameError: name 'origin_executor' is not defined
      9da615f1
  9. 09 Dec, 2022 6 commits
    • Jehan's avatar
      tools: various script improvements. · 15e8f85a
      Jehan authored
      - gimputils.version.find_latest() now gets data from the gimp_version.json file.
        We can't fully trust the 0.0_LATEST-IS* files anymore because they are on the
        download server and now all requests to it are mirrored (unless it's too soon
        and no mirrors are up-to-date). Mirrorbits proceeds with some health checks,
        but it's not enough to call them security checks as it goes through a separate
        rsync or FTP channels. The mirrors still have the ability to "lie" with
        targetted attacks. This is why our gimp-check-mirrors.py script is still
        useful.
        The json file on the other hand is fully controlled.
      - Also the second change in gimputils.version.find_latest() is that we can get
        the original checksums from the json file as well, instead of computing these
        ourselves. It's faster and safer. Since it changed the signature of the
        function, I also fixed its usage in download-plot.py.
      - gimp-check-mirrors.py uses the new version of gimputils.version.find_latest()
        as well. I also fixed the local path after changes in the mirror listing. The
        script was totally broken these days because of this.
      15e8f85a
    • Jehan's avatar
      devel-docs: add a bit of temporary pod creation command samples. · 138687b1
      Jehan authored
      I used these for the recent artfiles mirror debugging issues!
      138687b1
    • Jehan's avatar
      content: remove nbshare.io as mirror sponsor. · e3056542
      Jehan authored
      It's very weird as I had one answer about our mirror changes, early September,
      then nothing since, despite sending a few more reminder emails. We can't just
      ask forever, so let's remove them from the list for now.
      
      If ever they get back to us in the future, we can review and fix the settings to
      get them back in the mirror pool.
      e3056542
    • Jehan's avatar
      tools: simplify select to copy with pointer from terminal. · d1938567
      Jehan authored
      When double-clicking to select the --no-download-oc option, it was also getting
      the final dot, which I have to remove after pasting. Just make so we don't end
      with a final dot. It's not a huge issue, but that's the kind of tiny annoyances
      which add up to really annoy you in the end if you don't take care of them
      (especially if I often use these scripts, which have been the case these past
      months as we were setting up mirrorbits).
      d1938567
    • Jehan's avatar
      content, tools: 2 new FCIX mirrors. · 11ad3974
      Jehan authored
      See: Infrastructure#951
      
      Also added the contact field for artfiles.org (looks like I forgot to amend it
      in the previous commit).
      
      Finally update the downloads.http.txt generated file which was a bit outdated.
      11ad3974
    • Jehan's avatar
      content: Artfiles mirror is finally back in the download pool! · 57d98136
      Jehan authored
      Ever since the switch to mirrorbits, we had lost artfiles.org as a mirror,
      mostly because I never managed to get in touch with anyone on the email address
      which is in the rsyncd secrets file, neither on the info@ address I found for
      the artfiles.de hosting company.
      
      I finally got in touch with someone through the live chat on their website,
      which is therefore what I wrote down as official contact. They set up a FTP
      access for us, with credentials, and IP-restricted (in case GNOME change
      mirrorbits origin IPs, it needs to be remembered).
      
      It is now working in mirrorbits except I realized their mirror is outdated,
      which is the last point I'm handling with them by email right now (possibly they
      haven't changed from download.gimp.org to master.gnome.org?).
      57d98136
  10. 05 Dec, 2022 1 commit
  11. 02 Dec, 2022 3 commits
  12. 01 Dec, 2022 2 commits
  13. 21 Nov, 2022 3 commits
  14. 18 Nov, 2022 3 commits
  15. 17 Nov, 2022 1 commit