Is the forum slow for anyone else?

[Robert Ramsay]

I've noticed in the last few days that the forum intermittently slows to a crawl when I click on things. Just wanted to check whether anyone else was getting this, and whether @SkullTraill had noticed anything?

 

[SkullTraill]

I'm noticing it's running slow now at 8:08pm PST
And now at 8:10pm it's back to normal

Might not be something to do with WF servers in that case. Been monitoring resource usage and everything seems to be pretty good?

 

[ewiz]

Might not be something to do with WF servers in that case. Been monitoring resource usage and everything seems to be pretty good?

Could have been a hiccup from my isp since it only lasted a minute.

 

[SkullTraill]

Damn lol, over 75% of traffic still being blocked due to failing the challenge?? What the hell... perplexing.

 

[Aeternus]

OK. New manual anti-bot cloudflare rules are implemented. Please let me know if you still face slowdowns from now on. If this doesn't fix the issue, then the only remaining solution will be to upgrade the server which... I'll have to see about the cost (it's a lot). The new ruleset should also not affect PayPal payment notifications so I'm really hoping this finally fixes the issue.

Also let me know if you're having any issues/delays with payments/account upgrades.

@Robert Ramsay @Asteriskos @Tryp @ewiz @Aeternus @Morell please keep me updated about the slowdown situation.

---

In just a couple minutes:
View attachment 633

16k requests challenged, only 21.5% passed the challenge. At first it was around 8%, seems to be climbing as legit users meet the challenge but damn that's a lot of bot traffic being blocked.

At 8:08 A.M. EEST the forum works smoothly which, to be honest, if I would have accessed it back then at this time, it would have been significantly slowed.

16k requests... damn... absolutely nauseous

 

[Morell]

It is annoying to see claudfare on loading every page but in this settings it lets me through every time and the page works for me. (Using Avast web-browser... not feeling like changing it as everything works)

Maybe it should be kept on... No idea how else to protect from constant robo-activity.

 

[SkullTraill]

It is annoying to see claudfare on loading every page but in this settings it lets me through every time and the page works for me. (Using Avast web-browser... not feeling like changing it as everything works)

Maybe it should be kept on... No idea how else to protect from constant robo-activity.

Shouldn't be happening on every page unless you have some super strict security/sandboxing settings on your browser...

 

[Aeternus]

@SkullTraill I think I have an idea about how to prevent the bots from indexing, may not be that effective but, who knows.

The idea is as follows:

In the HTML of the forum pages (like those of the sections of The Order, The Library and so on and so forth), you use the noindex tags for the HTML codes of the pages, so that AI crawlers are forbidden from indexing the contents of those sections. Additional server-level AI crawl control tools like Cloudflare's bot management rules can also be helpful to actively block scraper signatures.

Cloudflare's bot blocking rules can be activated freely from the option "Block AI Scrappers and Crawlers" found in Security 》Bots section of the Cloudflare control panel.

 

[SkullTraill]

@SkullTraill I think I have an idea about how to prevent the bots from indexing, may not be that effective but, who knows.

The idea is as follows:

In the HTML of the forum pages (like those of the sections of The Order, The Library and so on and so forth), you use the noindex tags for the HTML codes of the pages, so that AI crawlers are forbidden from indexing the contents of those sections. Additional server-level AI crawl control tools like Cloudflare's bot management rules can also be helpful to actively block scraper signatures.

Cloudflare's bot blocking rules can be activated freely from the option "Block AI Scrappers and Crawlers" found in Security 》Bots section of the Cloudflare control panel.

Firstly, WF still needs to be indexed for search engines, so all the trivial noindex and robots.txt crawler prevention is out the window. We also don’t want to block all AI scrapers as the major ones are also useful sources of traffic. I am already using Cloudflare security rules as mentioned previously, within the capabilities of the free tier.

If you’d like to learn more please make a thread in the technology section, let’s leave this thread for slowdown reports.