Post 9mHR7X2WV9sPL1PC7s by eliotberriot@mastodon.eliotberriot.com
(DIR) More posts by eliotberriot@mastodon.eliotberriot.com
(DIR) Post #9mHR7X2WV9sPL1PC7s by eliotberriot@mastodon.eliotberriot.com
2019-07-29T11:49:51Z
0 likes, 0 repeats
So, today I'm starting to work on https://dev.funkwhale.audio/funkwhale/funkwhale/issues/890, which is the last of the 3 big features to be included in the 0.20 release.Context: - https://dev.funkwhale.audio/funkwhale/funkwhale/issues/890 (implementation ticket)- https://governance.funkwhale.audio/d/CIK2gzXG/report-option-on-everything (high-level design discussion)I want to do a thread on this to organize my thoughts, because the feature involves lots of work, on the UI, the API, the database, and I think it may be interesting to document this.The thread will likely last at least a month, until the feature is complete, and I'll post regular updates, so feel free to mute this thread if you're not interested :)Of course, feel free to ask any question you may have!Now, let's start!
(DIR) Post #9mHR7XIpWWv89b2DXk by eliotberriot@mastodon.eliotberriot.com
2019-07-29T11:57:47Z
0 likes, 0 repeats
So, let's discuss a bit about the feature. What we want is to have a reporting mechanism in Funkwhale to allow users to notify their mods of potentially problematic content.It's similar to the reporting features that already exists in Mastodon, but there are a couple challenges that are entirely new:- We want to allow anonymous reports, at least for takedown requests- We have a lot of content that is specific to Funkwhale. Artists, albums, playlists, radios, etc. We have to design a feature that works for all of them.
(DIR) Post #9mHR7XXidApWtm06ka by eliotberriot@mastodon.eliotberriot.com
2019-07-29T12:01:25Z
0 likes, 0 repeats
What we have here is a really big feature (a XXL or Epic in my terminology), for which implementation will take multiple weeks (or even, but I hope no 🤞, months).Why is it so big? Because there is work to do on:- The database- The API- The UI- The federationAnd because almost everything is entirely new, so we basically can't reuse anything from the current codebase.
(DIR) Post #9mHR7XltmSAlbkdQqu by eliotberriot@mastodon.eliotberriot.com
2019-07-29T12:11:01Z
0 likes, 0 repeats
The main problem when working on this kind of features is that each part of the feature is interacting with others.It makes it harder to roll-out the feature incrementally, which makes everything else even harder:- testing- reviewing- bugfixingSo, when I start working on such a feature, the first thing I try to do is identifying the work that is required by all the other tasks, and the interactions/dependencies between other tasks.I then start with the tasks that are the most depended upon.
(DIR) Post #9mHR7Y18rmMkN1lbc0 by eliotberriot@mastodon.eliotberriot.com
2019-07-29T12:17:05Z
0 likes, 0 repeats
It will possibly seem obvious to some of you, but given Funkwhale (and most Web apps) architecture:- The UI depends on the API- The API depends on the database/model- The model depends on the specs/design requirements- The spec/design requirements comes from our discussionsSince we already have specs here, I just need to move on to the next step in the hierarchy, so the model/database design.Generally speaking, if you don't know were to start and the feature you're working on requires to store new data, you cannot really go wrong by working on the model first.The difficult part is to ensure we don't forget use cases when designing the model, because it's the most cumbersome part to update.
(DIR) Post #9mHR7YMlZNfBS5ssJk by eliotberriot@mastodon.eliotberriot.com
2019-07-29T12:20:55Z
0 likes, 0 repeats
So, it's what I'll be doing today (and possibly this week). Come up with a suitable model for the feature, test it, ensure it work for our use cases.I'll probably mix this first implementation with at least some basic API endpoints and tests to validate that we can actually cover the core use cases with this model.Depending on how it goes, I'll then move on to the next steps (UI/Remaining API), but it's to early to talk about that yet ;)
(DIR) Post #9mHR7YdmY7H4IrqSq8 by eliotberriot@mastodon.eliotberriot.com
2019-07-29T12:23:36Z
0 likes, 0 repeats
Now, I'll be less verbose, and write more code ;)
(DIR) Post #9mHR7Ytjao2D6LJChk by eliotberriot@mastodon.eliotberriot.com
2019-07-29T12:34:01Z
0 likes, 0 repeats
I typically start with a test, like this.Then iterate to include the fields I've forgot, etc.
(DIR) Post #9mHR7Z9gdUnLtolwZM by eliotberriot@mastodon.eliotberriot.com
2019-07-29T12:38:28Z
0 likes, 0 repeats
This test does nothing at all, and I'll typically remove it entirely by the time the feature is released.However, it reflects the current state of the model design, and gives me confidence to iterate on more and more complex versions of the same model
(DIR) Post #9mHR7ZRlYHFyntENkW by eliotberriot@mastodon.eliotberriot.com
2019-07-29T12:43:18Z
0 likes, 0 repeats
After a few minutes, I can already realize that I forgot:- a uuid field, to avoid leaking information about reports via an integer primary key,- a federation_id (fid) field, to store a unique id for the report over federation
(DIR) Post #9mHR7ZimX0rrefByGu by eliotberriot@mastodon.eliotberriot.com
2019-07-29T13:59:26Z
0 likes, 0 repeats
Ok, I'm going to stop there for today, but if you're interested, this is what I have so far:https://dev.funkwhale.audio/funkwhale/funkwhale/compare/develop...890-reports-model
(DIR) Post #9mHR7ZwbhbvWLXf0oy by eliotberriot@mastodon.eliotberriot.com
2019-07-30T08:51:04Z
0 likes, 0 repeats
So, back to work. I'm rather happy with my initial model from yesterday afternoon, so I think today's focus will be on API representations :)
(DIR) Post #9mHR7aB8pZYL4cScTY by eliotberriot@mastodon.eliotberriot.com
2019-07-30T09:14:34Z
0 likes, 0 repeats
The main issue I'm facing right now is to come up with a representation/validation flow for API clients to submit a report associated to an arbitrary entity.I think I'll go with a payload like:```{ summary: 'This content is offensive', type: 'offensive_content', target: { type: 'artist', id: 42, }}```
(DIR) Post #9mHR7aVhb8026O52WW by eliotberriot@mastodon.eliotberriot.com
2019-07-30T14:04:29Z
0 likes, 0 repeats
That was a productive day for me. I now have:- a decent model layer- a basic API endpoint to submit reports, that supports both authenticated and anonymous users (in which case an email address is required)- a pod setting to control what report categories can be submitted by anonymous users. It means admins could receive takedown requests from people without accounts, but other kind of reports would require an account on the pod. Or they could disable anonymous reports entirely. - Logic to store a "freezed" version of the reported entity, so that mods can understand the report and get context even if the reported entity is deleted- New oauth scopes to control access to reports (reading submitted reports, submitting reports)Not too bad for ~700 lines of code. This includes tests, for all of this, and we reached the 1400 unit tests milestone, btw!If you want to have a look, I've opened a Merge Request here:https://dev.funkwhale.audio/funkwhale/funkwhale/merge_requests/848It went faster than expected, and I'll probably move on to the next step this week. I will likely focus on the moderator workflow to review and act on submitted reports.
(DIR) Post #9mHR7akwgSC0rfDDHc by eliotberriot@mastodon.eliotberriot.com
2019-08-01T09:04:43Z
0 likes, 0 repeats
I pushed one minor update this morning to the model layer. We needed an additional field to keep a track of who is the "owner" of the reported entity, so that mods can quickly find out if a given account generate a lot of reports.The MR is now ready for review at https://dev.funkwhale.audio/funkwhale/funkwhale/merge_requests/848, and in the meantime, I'm going to start working on the moderator workflow.
(DIR) Post #9mHR7b0XkSfZe2Vfay by eliotberriot@mastodon.eliotberriot.com
2019-08-01T09:10:44Z
0 likes, 0 repeats
This is the current workflow I have in mind (it may evolve):1. Someone reports something (a playlist, an artist profile, another user, etc.)2. The report goes into the moderation queue3. We send a notification to the mods (in the web UI and by email probably)4. A mod review the report and takes the necessary actions, if any5. The mod marks the report as handled, so it is removed from the queueI'll ignore #1, since I'm focusing on the moderator workflow. So let's talk about step 2 to 5.
(DIR) Post #9mHR7bHYjCHSUoTG7M by eliotberriot@mastodon.eliotberriot.com
2019-08-01T09:15:22Z
0 likes, 0 repeats
I don't expect any UI/API for #2, because it's something that can be done entirely automatically on the backend when a report is submitted.#3 should be rather small too, the biggest part being working on the email notification.But #4 is another story. From my (small) experience when doing support, you need a lot context to handle reports and tickets in an efficient and meaningful way.Providing this context to mods will likely be the hardest part of the task.
(DIR) Post #9mHR7bUfwQlx9UbjYu by eliotberriot@mastodon.eliotberriot.com
2019-08-01T09:25:35Z
0 likes, 0 repeats
What kind of context am I referring to? - Is it the first time the entity is reported? Are there other open reports from the same person? Concerning the same entity?- What were previous moderation decisions involving the entity? Who handled the reports and made the decisions?- Why is the entity being reported?- Is the entity new or old?- Etc.And building an API and, more importantly, a UI to provide this information, well it isn't easy.
(DIR) Post #9mHR7bir5i7BrTF3fE by eliotberriot@mastodon.eliotberriot.com
2019-08-22T09:08:03Z
0 likes, 0 repeats
Slowly getting back to work on https://dev.funkwhale.audio/funkwhale/funkwhale/merge_requests/848 after three weeks. It isn't easy, to say the least :D
(DIR) Post #9mHR7bvcKGK6V3DFYW by eliotberriot@mastodon.eliotberriot.com
2019-08-22T09:08:44Z
0 likes, 0 repeats
Basically, I need to integrate the reviews from Esteban, and move on to the next MR, maybe I can do that today?
(DIR) Post #9mHR7cBDOGnfHQVhrs by eliotberriot@mastodon.eliotberriot.com
2019-08-22T09:27:43Z
0 likes, 0 repeats
Thanks to @estebanm reviews and the test suite, it was easier than expected to pick up where I left, so I'm about to merge 🎉 Next step, mods UI!
(DIR) Post #9mHR7cQoSHHE3noABE by eliotberriot@mastodon.eliotberriot.com
2019-08-26T13:58:45Z
0 likes, 1 repeats
It took me longer than expected but I'm back working on this afternoon.I've put up together a few API endpoints for moderators to query, retrieve and manage user-submitted reports.This is really simple at the moment, because I want to start hacking on the UI, and gradually add the missing bits in the API from there.Current API code is available at https://dev.funkwhale.audio/funkwhale/funkwhale/merge_requests/866, I plan to start UI work tomorrow, and work on it for the rest of the week.
(DIR) Post #9mlyzXvAL47GKJJXSS by eliotberriot@mastodon.eliotberriot.com
2019-08-28T15:30:33Z
0 likes, 0 repeats
So, where are we at today?Well, I've got a pretty decent UI for displaying report information and mark them as resolved/unresolved.It's not perfect, of course, but all the info is here, as well as the ability to quickly search and filter reports by status.The difficulty is to provide relevant info for any report, since really different types of entities can be reported (accounts, artists, playlists, etc.)So, It's now *possible* to handle reports, as shown in the attached video.Some UI bits are missing, like the ability to add notes to the report, I'll probably work on that tomorrow.(corresponding Merge Request: https://dev.funkwhale.audio/funkwhale/funkwhale/merge_requests/866)
(DIR) Post #9mlyzY7DcFl0vgxAFE by eliotberriot@mastodon.eliotberriot.com
2019-08-29T09:49:49Z
0 likes, 0 repeats
So, I now have a moderation note system, with notes you can tie to any object (but only reports and accounts will be supported in the first iteration).Moving on to the note form
(DIR) Post #9mlyzYIuul7BVyQVTk by eliotberriot@mastodon.eliotberriot.com
2019-08-29T13:45:08Z
0 likes, 0 repeats
Here is what I have for the note taking part, I'm quite happy with the current result!We have a live testing session with Ginny tomorrow to ensure everything is usable from a moderator perspective, so I expect some changes though ;)
(DIR) Post #9mlyzYWk5MAqCqtY1o by eliotberriot@mastodon.eliotberriot.com
2019-09-02T14:22:32Z
0 likes, 0 repeats
So, we did the live testing on friday with @GinnyMcQueen and a few issues were identified, but overall, the UI was satisfactory.I spend today integrating Ginny's suggested changes, mainly :- More feedback when changing a report category- Remove the ability to delete a report- Add a confirmation before deleting a moderator noteI've also pushed a few additional changes that improve the moderator experience or were simply not implemented:- Add action button to quickly delete a reported object- Added action buttons to quickly block a reported account/domain - Make resolving a report collapse the report, instead of triggering a full reload, to be less intrusive and allow quick undo
(DIR) Post #9mlyzYhjQUxqkw2K9o by eliotberriot@mastodon.eliotberriot.com
2019-09-02T14:31:25Z
0 likes, 0 repeats
I'm not really happy with the size of the merge request, (~2000 lines of code added, mostly in front-end), but I don't think I could really have done better:It's a complex UI, with lots of new/specific needs, so I couldn't reuse that much in the existing codebase :/
(DIR) Post #9mlyzYt4kK2RK7LNq4 by eliotberriot@mastodon.eliotberriot.com
2019-09-02T14:47:18Z
0 likes, 0 repeats
Next stop: user-facing form/UI to submit reports.Once this bit is done, only small things will remain, like email notifications.Depending of other priorities, reports federation will be implemented in the 0.20 or 0.21 release.
(DIR) Post #9mlyzZ7FtbNg25yhwO by eliotberriot@mastodon.eliotberriot.com
2019-09-03T09:10:14Z
0 likes, 0 repeats
So, I'm starting to work on the user-facing report form itself.It's fun, because most of the time, I work on end-user (as in: non mod/admin) features first.
(DIR) Post #9mlyzZIxC6jqcNS3Au by eliotberriot@mastodon.eliotberriot.com
2019-09-03T13:44:29Z
0 likes, 0 repeats
Reporting yourself.It's not a bug, it's a feature.
(DIR) Post #9mlyzZX8LO55KM5NHE by eliotberriot@mastodon.eliotberriot.com
2019-09-03T13:57:31Z
0 likes, 0 repeats
But hey, joke aside, this is what I have so far.- A basic report modal that can successfully submit reports- An updated play button, meaning anything you can play, you can also reportSerendipity bonus: I spotted and fixed a completely unrelated N+1 query issue that affected performance when loading user activity.At this point, I need to work on the anonymous report thing, and plug the report option at various places in the UI, e.g for playlists, radios, libraries…
(DIR) Post #9mlyzZkFYcZZz2Dqim by eliotberriot@mastodon.eliotberriot.com
2019-09-06T11:47:53Z
0 likes, 0 repeats
So, worked on the end-user UI for reporting stuff, and I think this bit is is in a good enough shape to be reviewed.Code and can be found and reviewed at https://dev.funkwhale.audio/funkwhale/funkwhale/merge_requests/872, but you can also leave some feedback here of course :)
(DIR) Post #9mlyzZvwr7vkZJhBxI by eliotberriot@mastodon.eliotberriot.com
2019-09-06T11:55:00Z
1 likes, 0 repeats
Meaning I can work on the remaining pieces, which are much, much smaller and can be done in parallel :- moderator notifications (real-time in the UI, and by email)- rate limiting anonymous reports- small UI tweaks- support reports on libraries (we still lack the UI for this)Given theinpcoming deadline for the 0.20 release, I think I'm going to leave report federation on hold for a future release.So, I'm confident I can finish all of this next week and use the remaining time to implement/fix remaining tickets for the 0.20 release.https://dev.funkwhale.audio/funkwhale/funkwhale/-/milestones/18