Needle cleanup script, for unused/expired needles

A python script, used with the openqa tests and needles repositories, to identify, and optionally delete, unused and/or expired needles. It can also identify any missing needles used by the tests, and needles that use badly formatted datetimes, i.e not <needle_basename>-<datetime>.

When needles are deleted, a new branch of the needles repository is created and a commit generated for the deleted json/png files. The script can also generate an MR for this commit if the --create-merge-request flag is set, and an access token for the needles repository is provided using the --token option.

To use:

python needles_cleanup.py <path-to-tests-repo> <path-to-needles-repo> 

Optional arguments are:

--summary                 : Displays a summary of the needle states, and attempts no needle deletion
--expiry_date <YYYYMMDD>  : A datetime before which needles are considered expired
--branch <branch-name>    : The branch name of the needle repository to work on. Currently defaults to 'master'
--create-merge-request    : Use if you want cleanup to create an MR, in addition to the creation of a commit
--token <access-token>    : Gitlab access token, required to generate MR

On running the script with the following:

python needles_cleanup.py <path-to-tests-repo> <path-to-needles-repo> --expiry_date 20231201

A console menu will appear, provided there are unused and expired needles, like so:

What would you like to do?
[u] Delete unused needles and submit MR
[e] Delete expired needles and submit MR
[q] Quit
:

If a deletion choice is chosen, then a further menu will appear, offering the choice of bulk deletion or individually selecting needles for deletion, like so:

Deleting expired needles
(b)ulk delete needles, (s)elect individually, or (r)eturn to menu:

If individual selection is chosen:

Delete: app_baobab_home-20230423 (y/N):
Delete: app_baobab_home-20230712 (y/N):
Delete: app_baobab_home-20230819 (y/N):
Delete: app_gnome_calculator_home-20230423 (y/N):
etc ...

After selection, a msg containing the newly created branch with the delete commit in it is displayed:

Created branch: needles-cleanup/delete-expired-needles-1711027736,  with a commit for the deletion of the needles

Note: Each branch is uniquely identified with an epoch timestamp.

As mentioned above, if an access token and a flag set to create a merge request are supplied, then one will be created directly after the branch creation.