user agent filter - idea/proposal

2025-01-08 16:45:28

adminsforum@hubzilla.org

Einer von Vielen wrote the following post Wed, 08 Jan 2025 16:45:28 +0100

user agent filter - idea/proposal

I stumbled upon the complain about many server request by bots, see #^https://hub.volse.no/channel/harald?mid=c79e614b-2e82-46b0-99aa-ef420d15cafd
(can not comment there).

Are you aware of the streams #addon #uafilter #^https://codeberg.org/streams/streams-addons/src/branch/release/uafilter/uafilter.php

This could be a nice and useful addon for hubzilla as well. May be with a list of unwanted user agents that gets regular updates.

2025-01-08 16:56:03

*_jayrope

jrp@hub.kliklak.net

Sounds helpful. I do get cryptical comments sometimes, with bitcoin codes and such, from random-generated profiles, however blocking their servers doesn't seem to fully help there, neither does superblocking them (too tedious) - for the self-learning effect an addon like this should probably come with a feature, that enables showing the user-agent string as part of a post.

• sent from #Hubzilla at #Fediverse.

2025-01-08 16:57:24

*_jayrope

jrp@hub.kliklak.net

Eventually make it a request on the git -> #^https://framagit.org/hubzilla/addons

• sent from #Hubzilla at #Fediverse.

2025-01-08 19:29:13

Harald Eilertsen

harald@hub.volse.no

@Einer von Vielen I'm aware of it, but I would like something that would work regardless of what web app(s) the server is hosting. I know about some possible solutions that should block these requests early, but have not had time to test them properly yet.

However, I'm also thinking about a more adversarial response, where we could use these "AI" scraper bots' complete lack of respect for the sites they invade against them. What if we could serve them an alternative page/site with garbage that would poison their training data? Preferably something that's less resource demanding on the server than the actual web app.

Anyways just thinking out loud. Just blocking them is probably the easiest, but as they keep changing user agent strings and IP address ranges all the time, it'll be a game of whack-a-mole. I'd like something that does more damage.