Except in insert mode, deletions should zero-out data and not alter payload size
The documentation makes a good case for why it's good to not change the length of a file by default:
Unlike traditional text editors, having Insert Mode off is considered the default state in GHex, given that hex editors are routinely used to edit existing data in binary files that are very specifically laid out. In some cases, such files may be of a fixed size and increasing or decreasing their size may cause undefined or unwanted behaviour in applications that open them.
However, the delete key changes the length of the file by removing a byte and moving the rest of the file closer to the beginning of the file. Instead, what if delete just zeroed out the byte by default? If you want to remove a byte from the file and change the length of the file, then you could switch to insert mode and then press delete.