reproducible sync failure due to truncated 32KB manifest.xml file (gvfs using sftp)
Thanks for writing and maintaining gnote. I used tomboy for quite a while and am switching over to gnote after upgrading to a newer ubuntu release that dropped tomboy. Your efforts have saved me a lot of work.
OS: Ubuntu 20.10
Reproduced this issue on gnote versions:
- 3.36 (ubuntu package)
- 3.38.2 (gnome-3-38 branch) built from source
- 40.rc (master b4e5f660) built from source (with and without debug enabled)
I have 779 notes. These were imported from Tomboy automatically and seem to work just fine locally in any version of gnote.
I then followed this sequence (description is for 40.rc which does better with cleaning up the lock file in this error scenario):
- Create an empty directory on an ssh server
- Enable sync via gvfs using an sftp://user@server/home/user/Gnote URL to the new empty directory on the server
- Start the sync
- sync writes a
lock
file - sync creates a
0
directory - sync transfers all of my
.note
files (super quick in 40.rc nice!) - sync writes a
manifest.xml
file at the top level and in the 0/0 dir-
NOTE These xml files are truncated at exactly 32768 bytes and are incomplete (ie. they're not valid xml and are missing lots of
<note>
entries)
-
NOTE These xml files are truncated at exactly 32768 bytes and are incomplete (ie. they're not valid xml and are missing lots of
- gnote seems to read its own (truncated) xml file back and prints a bunch of errors (see below)
- sync deletes all of the
.note
files from the remote server - sync deletes the lock file
- gnote says everything is sync'd OK and waits for me to close the dialog
Here is a (slightly redacted) version of the DEBUG output during the sync. The errors are all pointing to the end of the truncated file and complaining about the xml being invalid.
$ gnote
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_from_file - Can't read copyright, using none: Key file does not have key “Copyright” in group “AddinInfo”
(140686276463232) DEBUG: load_module - load module libbacklinks
(140686276463232) DEBUG: load_module - load module libfilesystemsyncservice
(140686276463232) DEBUG: load_module - load module libfixedwidth
(140686276463232) DEBUG: load_module - load module libgvfssyncservice
(140686276463232) DEBUG: load_module - load module libprintnotes
(140686276463232) DEBUG: load_module - load module libreplacetitle
(140686276463232) DEBUG: load_module - load module libunderline
(140686276463232) DEBUG: load_module - load module libwebdavsyncservice
(140686276463232) DEBUG: parse_tags - found tag system:template
(140686276463232) DEBUG: update_sync_action - Autosync pref changed...restarting sync timer
(140686276463232) DEBUG: RemoteControl - initialized remote control
(140686276463232) DEBUG: register_object - Gnote remote control active.
(140686276463232) DEBUG: needs_execute - needs execute?
(140686276463232) DEBUG: perform_synchronization - Creating synchronization thread
(140685932967488) DEBUG: synchronization_thread - SyncThread using SyncServiceAddin: Online Folder
(140685932967488) DEBUG: synchronization_thread - 8
(140685932967488) DEBUG: synchronization_thread - Sync: GetNoteUpdatesSince rev -1
(140685932967488) DEBUG: get_note_updates_since - get_note_updates_since (-1) returning: 0
(140685932967488) DEBUG: synchronization_thread - Sync: 0 updates since rev -1
(140685932967488) DEBUG: synchronization_thread - Sync: Uploading 779 note updates
(140685932967488) DEBUG: upload_notes - UploadNotes: notes.Count = 779
sftp://user@server/home/user/Gnote/manifest.xml:2: parser error : AttValue: ' expected
="0"/><note id="5fbd008c-7234-4422-9014-916a5d155682" rev="0"/><note id="559b41f
^
sftp://user@server/home/user/Gnote/manifest.xml:2: parser error : attributes construct error
="0"/><note id="5fbd008c-7234-4422-9014-916a5d155682" rev="0"/><note id="559b41f
^
sftp://user@server/home/user/Gnote/manifest.xml:2: parser error : Couldn't find end of Start Tag note line 2
="0"/><note id="5fbd008c-7234-4422-9014-916a5d155682" rev="0"/><note id="559b41f
^
sftp://user@server/home/user/Gnote/manifest.xml:2: parser error : EndTag: '</' not found
="0"/><note id="5fbd008c-7234-4422-9014-916a5d155682" rev="0"/><note id="559b41f
^
sftp://user@server/home/user/Gnote/0/0/manifest.xml:2: parser error : AttValue: ' expected
="0"/><note id="5fbd008c-7234-4422-9014-916a5d155682" rev="0"/><note id="559b41f
^
sftp://user@server/home/user/Gnote/0/0/manifest.xml:2: parser error : attributes construct error
="0"/><note id="5fbd008c-7234-4422-9014-916a5d155682" rev="0"/><note id="559b41f
^
sftp://user@server/home/user/Gnote/0/0/manifest.xml:2: parser error : Couldn't find end of Start Tag note line 2
="0"/><note id="5fbd008c-7234-4422-9014-916a5d155682" rev="0"/><note id="559b41f
^
sftp://user@server/home/user/Gnote/0/0/manifest.xml:2: parser error : EndTag: '</' not found
="0"/><note id="5fbd008c-7234-4422-9014-916a5d155682" rev="0"/><note id="559b41f
^
(140685932967488) DEBUG: synchronization_thread - Sync: New revision: 0
(140686276463232) DEBUG: on_exiting_event - Saving unsaved notes...
I suspect that this would be reproducible with any manifest that exceeded 32768 bytes in xml form but haven't experimented enough to be sure yet. Wanted to get this reported ahead of your 40 release.
Each note takes up 58 bytes in the manifest (eg.):
<note id="4d1b427c-c17f-4ffa-8e9f-f8d88c5dc7b1" rev="0"/>
My manifest.xml
file would need to be > 45182 bytes once the other misc xml tags were added.
I'm able to build and test patches and have backups of all of my notes. Let me know if I can help you with debugging.