Skip to content

Draft: OpenSearch search engines and search suggestions support

Vanadiae requested to merge Vanadiae/epiphany:opensearch into master

(also includes the migrator commit from !1052 (merged), but the MR will be rebased on master once that MR is merged) Done.

TODO/What is done or works:

  • Downloading, parsing and adding an Opensearch engine works just fine on the EphySearchEngineManager. The search engine works fine for searches, and shows up properly in the preferences.
  • The opensearch code takes 400 lines in the main manager code (almost half of it). Port manager to GObject w/ GListModel and the list box with it. The API will change and be cleaner. Then split the opensearch code in a separate lib/ephy-opensearch.c file. => port is in !1055 (merged)
  • While the code for opensearch links autodiscovery seems fine, webkit gives a WebKitJavascriptError without any error message nor in the GError nor in the web inspector console. Javascript alone runs fine when entering it manually in the web inspector console. Need help as I'm out of idea here. => It seems like the Node objects stored in the returned Array cannot go through JSC boundaries. I was able to pass a dummy object, which hence made it clear that I only had to map the Node objects to plain objects, with the same node properties I care about (href and title). See ephy.js at 04632385.
  • While the suggestions URL is already properly parsed and collected when adding a search engine, there is no suggestions support in the location entry yet. Search suggestions should also work when using the bang (so that typing things with a bang at the beginning will suggest using the appropriate search engine, not the default one).
  • The popover needs design work. And a new icon for the button. Also needs to give feedback when the search engine is successfully added, or otherwise the error in the popover.
  • While it's nice to have an icon for OpenSearch support, it's better if it became like "glowing" (same as the focus outline in GTK4) when we detect lots of searches (> around 20) are made manually on the website, to encourage people to add search engines (also advertise when suggestions support is present). Want to be careful with alert fatigue, and to be careful with privacy settings too. Given how simple Epiphany's UI is, the "loupe" icon in the URL entry should be sufficiently noticeable.
  • On the same idea, encourage people to define a bang shortcut (and give them an entry to type one when adding an opensearch engine, or bring them to the added engine in the prefs).
  • Explain in the prefs that bangs can be used for search suggestions too, and show something (an icon w/ color + descriptive text?) to let know whether a particular search engine supports search suggestions.
  • Also, it would be really awesome if Epiphany could show up a dialog when it starts up (only the first time ofc) to list the search engines that could be replaced with ones coming from OpenSearch files, if they have suggestions support (since otherwise people wouldn't get any additional feature so it would be useless/annoying/confusing). One way of implementing this would be to simply to a sample search (i.e. "example" as search query) with each search engines, and to try autodiscovering an opensearch description file on the page we get sent. If the engine has suggestions support, then we add it to the list of search engines that can receive the search suggestions feature. In that case we'd only keep the opensearch_url and suggestions_url from the OpenSearch file, and keep the original ones from the EphySearchEngineManager. => WIP
  • Drop Google search suggestions support in favor of default Google engine w/ opensearch/suggestions (opensearch_google.xml, just adapt and insert it as default value in the gschema).
  • Ask Google to make the referrer:source change as done ^^, and ask them to include an autodiscovery link too if possible. => they never seemed to have the autodiscovery link (I checked on the Wayback Machine), and anyway I just won't bother with that.
  • Should the GNOME Shell search provider also provide search suggestions? This could be a nice feature, but I'd be much in favor of disabling it by default for privacy reasons when searching from GNOME Shell, with eventually a preference for it. It's also probably best to not allow at all (not even with a preference) search suggestions done without a bang (i.e. with the default engine) with the search provider.
  • What about search suggestions in private window mode? Firefox disables search suggestions both for "default engine" searches and bang searches. I can understand the choice of disabling search suggestions when no bang is used, but with a bang it seems unnecessary and annoying to disable them.
  • Scrolling to the just added engine works when using the "Add" button directly from the preferences, but it does not work properly when adding an opensearch one, probably because of the animation of the preferences window or something: it only gets scrolled until the expander row itself (the heading part) is at the bottom of the prefs window. I didn't figure out where the window animation/transition code or CSS lives, and anyway I don't see a way to fix it.
  • Figure out why the favicon code isn't smart enough to figure out the icon for search suggestions (unless a particular search suggestion URL has already been visited) => it should just pick the site-global one if it doesn't have one for this exact URL. WebKit does all the hard work so there's not much I can do on my side
  • Support <Param> in <Url>
  • Drop all debugging/testing code
  • Fix all fixme/todo
  • Write an actual commit message
  • add tests for all new opensearch code
  • Update to latest libadwaita branch commit and get the libadwaita MR merged, then remove its subproject module in the flatpak manifest here and drop developer_mode=true in meson_options.txt
  • Add a privacy preference for suggestions support
  • Do not try to lookup suggestions if the text looks like an URL or URL-like (i.e. gnome.org). We really shouldn't be "leaking" all URLs like that.

Fixes #1496

Depends on libadwaita!852

Edited by Vanadiae

Merge request reports