Mastodon Feed
Boosted by jsonstein@masto.deoan.org ("Jeff Sonstein"):
dumpsterqueer@superseriousbusiness.org ("tobi is writing bugs :terminal_cursor:") wrote:
hey @dansup, I had a look at fedidb the other day and noticed a few oddities it might be worth you and/or the team taking a look at:
- numbers are constantly going down, I believe because you're wrongly omitting instances that have an open robots.txt file, and so you're probably dropping entries from your database (https://github.com/fedidb/issues/issues/55). This gives the misleading impression that the fedi is rapidly bleeding out (see screenshot), but that's probably just because there's something wrong with the scraper / database.
- some softwares are missing entirely, like snac and gotosocial (https://github.com/fedidb/issues/issues/58, https://github.com/fedidb/issues/issues/49); in the latter case it was my understanding you'd added gts again after you added proper robots.txt support (https://mastodon.social/@dansup/114023569148179068) but it seems it was quietly dropped again at some point
- loops.video shows exactly the same mau as it does registered users, this seems like a bug as it's basically impossible for that to actually be the case, and it puts loops.video way higher in the "top servers" list than it probably should be
are there plans to fix these issues? It's my understanding that you've been working on fedidb stuff again because you posted some screenshots a whole ago iirc, but in the meantime perhaps it's worth pausing scraping again like you did a few months ago, and adding a notice to the top of the page that work is in progress to fix things?
one reason I'm asking is that currently on every gts instance homepage with closed registrations we link to fedidb as a "this instance is closed but you can use these tools to find an instance that is accepting registrations" sort of thing, but it's not great to link to a website that doesn't show GtS at all and has stats that look a bit bananas
