Attachments
|
|
| dodgy_amp.har |
|
|
|
|
| dodgy_amp.har |
Activity
2019-05-10 23:03:55
The opening is followed by some CSS and then a whole load of obfuscated javascript (see screenshot).
The important things of note though are
- All that's one one line (yeuch)
- Google aren't actually declaring the document as being AMP (which is why the detector isn't firing) - note the
2019-05-10 23:03:55
2019-05-10 23:07:41
2019-05-10 23:07:41
2019-05-10 23:16:46
Webhook User-Agent
View Commit
2019-05-10 23:25:58
Webhook User-Agent
View Commit
2019-05-11 09:52:44
Looking at logs it's because there are a couple of hops to make:
- Start: https://www.google.com/amp/s/www.theregister.co.uk/AMP/2017/05/19/open_source_insider_google_amp_bad_bad_bad/
- Go to listed canonical (still AMP): https://www.theregister.co.uk/AMP/2017/05/19/open_source_insider_google_amp_bad_bad_bad/
- Go to listed Canonical (proper HTML): https://www.theregister.co.uk/2017/05/19/open_source_insider_google_amp_bad_bad_bad/
2019-05-12 09:08:24
I've had no luck (user-agent changes, incognito + UA change etc) in getting Google to actually serve me AMP results so haven't been able to repro.
2019-05-12 09:10:18
The URL is the same as those mentioned earlier, but when the user goes there direct the redirect works.
2019-05-12 09:10:18
2019-05-12 09:35:21
I've now been able to get Google to serve me AMP results. But... the must be doing some kind of compatability or experience tracking on their side, because if you do the following:
- Open Private Window in FF
- Select UA that wouldn't get AMP results
- Search "theregister AMP bad"
- No AMPs
- Change to UA that should, re-search
You still get no AMP results.
This is important because it seems that the default Iphone user-agent in the User-Agent switcher extensions, for whatever reason, doesn't get given AMP results. So when I later switched to Mobile/Chrome in there, I still didn't get the results even though I should have. Closing the private window, and then opening anew and starting off with the correct UA gets me AMP results.
Unfortunately, the behaviour I'm getting still differs to the user. I may need to lay hands a Mac to try and repro with Safari. I don't see any obvious smoking gun in the console output - the call out to
It's a blind shot in the dark, but I'm tempted to get the user to run a copy of the script which adds that domain to the checker, just to see what the behaviour ends up being.
Either way, we already know the initial change won't be sufficient on it's own, as both Cloudflare and Bing run amp caches too. The problem there, though, is they're rolling out "Real URL" support, so we won't be able to rely on the source hostname in future as they'll serve AMP from
Though, to be fair, based on a very quick check, Cloudflare do seem to be honouring the spec and properly declaring AMP
2019-05-12 09:49:19
So, we've got google's wrapper page, which basically involves serving a small document with a lot of javascript in it (obfuscated in the way most of Google's JS is).
Worth noting, for clients without javascript, there's also a
But, ultimately, Google's page just results in a page which has an iframe in it referencing the actual amp cache.
So, if we go to https://www.google.com/amp/s/www.theregister.co.uk/AMP/2017/05/19/open_source_insider_google_amp_bad_bad_bad/ the iframe loads content from
If you try and access that direct in a browser, you'll get a 404, need to make sure that every request header is correct
At which point, you get our source document
Neither of those pages properly declares themselves as being AMP.
Now, if we try and detect the AMP hostname, that'll trigger from within the iframe, and so won't be able to redirect the browser.
The bit I still can't fully answer though, is why the user's Safari isn't triggering as the wrapper domain is (or, at least, should be) correct for the check that was added earlier in this issue. Need to look at that screenshot again, there must be something I'm missing.
2019-05-12 09:57:54
That directive was added because otherwise the anti-AMP protection wouldn't work on sites with moderately strict Content Security Policies. See here - https://projects.bentasker.co.uk/jira_projects/browse/MISC-25.html#comment2186467
There is a complaint about
I should be able to test this to reproduce by creating a new user script which injects the code without using
2019-05-12 10:08:49
The result isn't actually quite as expected.
What I got was
There are, as expected the CSP violations, but they both actually originate from the document within the iframe rather than at Google's level. And actually, if we look at the response serving that wrapper, they're not serving a CSP.
So, given the CSP violation occurs within the iframe, it shouldn't stop the redirect from happening (the redirect is disabled in my test userscript and just calls
2019-05-12 10:22:23
Wait.... it's limited to
Anyway, means we can disregard the CSP related lines in the user's screenshot
2019-05-12 10:25:36
To try and get the user up and running though, I'll create a minor release with an iframe check in it. Won't update the hook to refer to it though as the performance overhead will likely be quite high (as we'll need to do a full DOM scan).
2019-05-12 10:34:45
2019-05-12 10:44:46
Webhook User-Agent
View Commit
2019-05-12 10:47:00
Rolled out onto the CDN here - https://github.com/bentasker/adblocklists/commit/d3a8cd0399bb361f250f5e308dbe4e917dc14836
Just need to calculate the SRI hash:
2019-05-12 11:05:58
He's noted that if he adjusts the hook script to run
2019-05-15 13:55:20
2019-05-15 13:55:20
2019-05-15 14:35:56
Steps to repro (hopefully)
- Be using Safari
- Install Tampermonkey (tampermonkey.net will take you to the relevant safariextensions page)
-
- Go to https://github.com/bentasker/RemoveAMP/raw/master/greasemonkey_hook.user.js
- When tampermonkey prompts, press
-
- Google "The Register Amp Bad"
- Results should have the Register article at the top with a little Amp icon
-
- Click
- Click the link to the El Reg article
- Redirect doesn't trigger
2019-05-15 14:37:04
2019-05-15 14:47:02
2019-05-15 14:58:19
Although the address in Safari's address bar is https://www.google.com/amp/s/www.theregister.co.uk/AMP/2017/05/19/open_source_insider_google_amp_bad_bad_bad/ we never actually see a request for the path
Looking through the HAR with
So, it seems Google aren't reloading the page, they're simply rewriting it with JS and using JS to update the address bar (AMP and address bar tampering, two things I loathe at once.... fun).
2019-05-15 15:10:06
I've not bothered digging through Google's JS to try and find what reads what, but it does seem that if we tamper with their attributes it won't use Google's internal page and instead appears to go straight to
I just want to take a quick look though, and see whether we can disrupt even that and get taken direct
2019-05-15 15:23:26
var da; var bads = ['data-amp','data-amp-cur','data-amp-title','data-amp-vgi','ping']; var eles = document.getElementsByClassName('amp_r'); for (var i=0; i<eles.length; i++){ if (eles[i].tagName.toLowerCase() != 'a'){ continue; } da = eles[i].getAttribute('data-amp-cur'); if (! da){ continue; } eles[i].href = da; for (n=0; n<bads.length; n++){ eles[i].removeAttribute(bads[n]); } }The challenge, of course, will be working out how best to trigger it. My inclination to begin with, though, is just to create a new Greasemonkey script for it, limited to the Google pages and try that
2019-05-15 15:30:46
Webhook User-Agent
View Commit
2019-05-15 15:39:31
The attributes correctly get purged, and the
Fuck you Google. It doesn't look like it gets appended until you actually click the link. I guess hacky will have to do for now.
2019-05-15 15:44:45
Webhook User-Agent
View Commit
2019-05-15 15:48:28
The problem I have is that it may prove to be quite fragile. We're already reliant on Google not changing the class name
Will take a quick look at how their ping URLs work to see if we can use that instead. I've a feeling though, that if
2019-05-15 15:51:48
Makes sense really otherwise they'd have people bouncing victims of their redirect page onto other locations.
OK, will leave as is for now
2019-05-15 16:04:24
2019-05-15 16:04:24
2019-05-15 16:22:46
Webhook User-Agent
View Commit
2019-05-15 16:27:36
I've rolled this as v1.4.1 - https://github.com/bentasker/RemoveAMP/releases/tag/v1.4.1
Marking as fixed
2019-05-15 16:27:41
2019-05-15 16:27:41
2019-05-15 16:27:45
2019-06-09 12:36:43
2019-06-09 12:36:43
2019-06-09 12:36:43
2019-06-09 12:36:43
2019-06-09 12:36:43
2019-06-09 12:36:43
2019-06-09 12:36:43
2019-06-09 12:36:43
2019-06-09 12:36:43
2019-06-09 12:38:36
- MISC-25 becomes FKAMP-1
- MISC-29 becomes FKAMP-2
- MISC-31 becomes FKAMP-3
2019-06-09 12:38:44
2019-06-09 12:38:44
2019-06-09 12:38:51
2019-06-09 12:38:55
2019-06-09 12:38:55
2019-06-09 12:39:00