As part of the next XMB version in development, I made changes to help with the new pricing structure of Google's reCAPTCHA service.

In our original implementation, the captcha was displayed as soon as anyone visits the registration page. This causes unwanted traffic between bots and the Google API.

Going forward, there will be an introduction page for registration. This means the reCAPTCHA gets pushed back to the 2nd page and won't be visited by bots quite so frequently.

There is also a future change regarding keys "to be migrated to a Google Cloud projected by the end of 2025." It is my next priority to learn how that affects us here.

What I've learned so far is that it's not just about the keys. I migrated one of the 2 keys used here and that transition is seamless.

After that, there are a bunch of steps for actually switching APIs from the "siteverify" to the "enterprise" version. Here's how that might go:

https://cloud.google.com/recaptcha/docs/using-features

The reCAPTCHA keys will have to be traded for Google Cloud API keys. Of course, there's a whole new way of doing things behind the scenes too:

https://cloud.google.com/recaptcha/docs/create-assessment-we...

Here is the old API reference:

https://developers.google.com/recaptcha/docs/display

So this looks like a need for more settings, a way to switch between the two API implementations, and a lot of testing after it's ready.

Also consider an alternative, https://www.cloudflare.com/application-services/products/tur...

I'd rather avoid captchas altogether, they just annoy people while doing little to actually prevent spammers.

My proposal is to go for some user-configurable Q&A,. Then again, good luck educating admins to bother setting it up properly so it's both accessible enough for users and not too easy for spammers. (

I remember setting up a phpbb board over 4 years ago that I've long since left, and there was a 'sortable q&a' feature that had minimal JS and made you put different things into different boxes:



I think there's maybe been one spam account that registered in the whole time of its existence, and while it's... obnoxiously opinionated, to say the least, it works for the purpose of that specific forum.

In other words, configure something like this correctly, and you've got yourself something that works with minimal JS and tracking things - therefore works even under 'strained conditions'. You could expand on the idea a bit more than phpbb did which I think only allows for the one question - as long as the suggested defaults are strong I think admins should be okay

If there's an open source solution that you know about, that's possible. This isn't a wheel that we should try to reinvent.

I saw a site today using this one:

https://www.hcaptcha.com/

These different vendors have interesting free features. I'm just not sure yet if we need to support more than one.

Quote: Originally posted by miqrogroove

I saw a site today using this one: https://www.hcaptcha.com/ These different vendors have interesting free features. I'm just not sure yet if we need to support more than one.

I've seen numerous sites using that one. Might be worth adding since it's probably the one I encounter most after reCAPTCHA, and it seems to have especially come about ever since the changes to the API.

I'm reading more about these and have some new thoughts:

reCAPTCHA doesn't specifically require a migration to the new API. What they are really going to require is a new pricing structure. So I'm seeing a need to install the alpha version of XMB and find out if those updates will cut the reCAPTCHA traffic counts to an affordable level. If not, I would be inclined to move away from Google on this feature rather than implement a new API.

Cloudflare seems more committed to offering a free tier and eliminating user puzzles. This is my fallback plan.

hCaptcha is a little weird in the details. They offer a free tier, but they want $140 per month for passive verification. This puts them at a huge disadvantage.

I've never had issues with the cloudflare verification thingy. It seems like it might be the way to go.

I'm monitoring the bot traffic this morning and the results are interesting. Even though the bots are requesting captcha at a rate well in excess of 10,000 per month, the success rate is much lower. The XMB server is only seeing the verification/assessment process at a rate of about 3,000 per month.

The last thing to tackle for reCAPTCHA specifically is the new settings offered by Google. During setup, on the reCAPTCHA Classic site, it asks for "Score based (v3)" or "Challenge (v2)" type protection. Currently, we only support v2. And then it asks for Checkbox vs. Invisible, and we're not set up for the invisible mode. I also suspect there's no way for XMB to check which settings were used on the Google side of the API.

On the Google Cloud site, it doesn't ask for those things during setup. In step 2, there's a slider labeled, "Will you use challenges?"

I don't even know which combinations of these different sites and APIs are currently compatible with XMB. We will need to do some experiments and offer some instructions.

Quote: Originally posted by miqrogroove

I'm monitoring the bot traffic this morning and the results are interesting. Even though the bots are requesting captcha at a rate well in excess of 10,000 per month, the success rate is much lower. The XMB server is only seeing the verification/assessment process at a rate of about 3,000 per month.

Any specific patterns with the success rate? Countries, browser agents etc?

Could be worth looking into for the sakes of analytical purposes. I'm a little curious if anything crops up in particular.

I don't want to advertise the details too much so let's just say the latest changes are having a big impact on the captcha traffic. Officially, Google only publishes daily numbers, so after a few days I can share a screenshot of what happened in the daily trend.

Also remember, these changes tend to be temporary. The bots are always evolving so what works today isn't going to work years later.

I'm also weary of Google's intent to either monetize the bot traffic or "fail open" when quotas are exceeded. That's bad policy. I will continue to evaluate the idea of offering the Cloudflare alternative.

Understandable, don't want to give people ideas I suppose.

One thing about reCAPTCHA I have noticed that it treats you much more leniently if you're using latest Chrome/Edge/Safari and are logged into a Google account (and I presume it uses the data from that to determine whether to give a challenge or not). This, I'll be honest, is quite the concern for user privacy...

Personal anecdote - there are times I have recalled having to switch to my phone simply to fill out a reCAPTCHA because it's temporarily "blocked my network" - but then suddenly doesn't care when it's iOS Safari...

CloudFlare on the other hand doesn't seem to discriminate in this way.

Captcha requests on 6/26: 1,695

Captcha requests on 6/29: 2

LOL nice, literally just from moving it to the other page?

Btw, two of the links in the who's online thing for looking up IPs are broken, IIRC

Quote: Originally posted by flushedpancake

Two of the links in the who's online thing for looking up IPs are broken, IIRC

I can't confirm that. Please be less vague.

I tested it:

W
https://whois.domaintools.com/ip
requests validation

T
https://www.net.princeton.edu/cgi-bin/traceroute.pl?target=i...
not found

L
https://mxtoolbox.com/SuperTool.aspx?action=ptr%3aip
looks up ptr record

M
https://www.iptrackeronline.com/?ip_address=ip
limited to 4 lookups a day


Please consider ipinfo; cloudflare uses them for geolocation ip data
community.ipinfo.io/t/ipinfo-is-the-ip-geolocation-data-provider-of-cloudflare/6841

In an ideal world, forum admins could add 3 or 4 ip data providers of their own choice in the admin panel as providers seem to change their access or remove altogether.

Even the labels seem wrong. W and T seem to be the only ones that have meaning.
I guess you could always send in a github pull request. I'm cleaning up that bbcode doc file atm.

I might have to delete the traceroute link. Have you found any alternative?