[Proposal] ODRS changes
It looks like that since a while, many developers are not totally happy with the current experience with the reviews and rating system.
With this issue I would like to propose some change and if the idea gets positive feedback I'm available to take in charge the development of the changes.
ℹ ️ An introduction to this topic has been published by @cassidyjames on his personal blog. I suggest to read it before proceeding.
In this issue I'm not entering in technical and implementation details.
App rating
Drop the stars system.
It's really difficult to have a review from an average happy user but is really easy to get a negative review from users that are simply "not so happy". A simple up/down vote system along with the downloads count would clarify the weight of the current evaluation. For example 100 downvotes over 1'000 downloads and 100 downvotes over 10'000 downloads have a different meaning while (comparing to the current system) 3,5 stars on 1'000 downloads and 10'000 downloads have the same meaning as you don't know immediately how many people vs downloads have given a review.
Votes should record the app version used by the user. This data could be used as KPI and eventually as a filter.
Reviews
I would not require a comment (review) to be written in order to vote. Many people skip (mainly positive) rating because they don't know what to write in the free text box or even simply in the title box. A mandatory comment may also result in a quantity of identical comments.
Reviews should record the app version used by the user in order to clarify that an issue/bug may have been solved (this needs a bit of UI study). The stores may show old reviews in a specific format and/or don't show at all outdated reviews.
Reviews moderation
The main lack is actually the moderation part: devs have no idea of how to moderate and manage the reviews and how to control sh!tstorms. When you visit https://odrs.gnome.org/login you see You can request an account if you think it is required.
but still have no idea of what you can do.
The website's main page clearly states: we care very much what comments are shown to paying customers we definitely want reviews to be pre-approved and checked [...] For admin_distros like Fedora we don't have this luxury and so we're going to rely on the community to self-regulate reviews.
It would probably make sense to create a commitee like we have for many other topics.
Moreover I would create/update a management portal for:
- Self registration
- A Control Panel where you can claim/register your apps
- The moderation team would verify that who claimed/registered an app is actually authorized before that dev can do any operation (maybe a GitHub/GitLab integration would simplify the process?)
- An area where to see all the reviews (of the owned apps) and moderate them (first level moderation)
- A dashboard with statistics about the reviews of the owned apps
User identification
Allow (not force) users to authenticate when reviewing. The current identifier is a SHA1 hash of the machine-id and the UNIX username along with a salt
, meaning that haters can spin up many virtual machines and post many negative comments with no actual value.
Authenticated user's reviews may be identified in the reviews list to provide an added value.
Distribution package and store identification
The review should record from which store is coming and for which distribution package. The API should then allow to filter the reviews for the in-use-store.
It happens that negative reviews are not due to the app itself but due to how it has been packaged by 3rd parties, the Control Panel should allow to add team members and/or assign the moderation of a distribution package reviews to a specific (group of) user.
System specifications
Like Steam does, ask the user to share (anonymous or not depends on if the user is logged in) the data about the system specifications (CPU model, RAM type and amount, disk(s) size(s), etc). Would be useful to obtain the version of the dependencies declared by the app in order to have a more clear idea of the system's status.
Distribution identification
Allow the software stores to specify the distribution's name as flavours/spin-off/etc don't always update the distribution's name at system level.
Flathub
Flathub is planning to improve the "front page" experience on Flathub by encouraging better app metadata, but has avoided using ODRS so far because of some of these issues. User signals are a powerful data point, so it would be a great asset for any app store but specifically Flathub could really use this.
Editor's choice and other rewards
Using the Apple Store and Google Play as an example, use the reviews data along with the Editors (moderator committees?) experience to declare apps as the Editor's and/or User's choice in order to provide appreciation to developers who are working hard to make their users happy.
The downloads count and reviews count+value in a timeframe could work to automagically identify an app as emerging, as a rising star and stuff like that.
Bug reporting
Non technical users need a way to easily report bugs and issues in the app. Asking them to go on a git/project platform (GitHub, GitLab, BugZilla, YouTrack, whatever) leads to not shared feedbacks that may be important for the app's quality. At the same the management platforms may be overwhelmed of bug reports that are: not bugs, incomplete, not followed up, generic help request, etc.
This bug reporting could also jump into the statistics feature as a KPI for the app.
Moderation commitee volunteers
A bunch of people that is already willing to be in the moderation committee: @pietrodc, @cassidyjames, @mirkobrombin , @robertgzr , @bertob
Calls for volunteers should be opened.