MISC-35: Change Phone Search Engine to Ecosia



Issue Information

Issue Type: Task
 
Priority: Major
Status: Closed

Reported By:
Ben Tasker
Assigned To:
Ben Tasker
Project: Miscellaneous (MISC)
Resolution: Done (2019-09-07 10:14:28)

Created: 2019-08-12 23:25:40
Time Spent Working


Description
want to try and reduce my exposure to Google a bit.

Ecosia have a reasonable privacy policy - https://info.ecosia.org/privacy

They plant trees using ad revenue, but that's not a consideration here, realistically they probably won't get much past my ad blockers - however privacy orientated they may be, malvertising is still a real concern.

My biggest concern really is that they rely on Bing. There's a reason Google's still my primary SE, and that's the quality of results. Though, Google has been slipping recently...

So, the plan is to make it the default on my phone for a little while and see how I get on, and then perhaps look at swapping it in in other places


Issue Links

Ecosia.org
Toggle State Changes

Activity


Fairly straightforward in Firefox for Android:

- Go to ecosia.org
- Long press on the search box, Add search Engine
- Settings, Search, Ecosia, Set as default

Doesn't look like this version of Chrome on Android will let you add new providers. Have changed to DuckDuckGo so that I've not got a lazy route back to Google (well, aside from typing an URL in)

To be fair, I don't make much use of Chrome on Android anyway as the user experience just gets progressively worse with each release.
Time'll tell, I guess, how much use Ecosia actually is as an alternative.

If it goes well, there's a small possibility I may forget all about this ticket in the meantime. Conversely if it goes poorly it's likely to remind me to come back and update...
btasker changed Project from 'STAGING' to 'Miscellaneous'
btasker changed Key from 'STGNG-12' to 'MISC-35'
Long way from being ready for that yet, but just to note for the future:

Been tinkering around and it supports the site: operator, so could potentially use it to create a site specific search (though there is a small risk the user'd erase the site limiter if they try to refine the search terms).

Might be worth thinking about for domains that don't lend themselves to being easily searchable - projects.bentasker.co.uk being a prime example.
Now that I'm on my desktop, taking a quick look at what Ecosia pulls in/loads. Ad blocker is currently enabled, to keep the noise down.

On the Search landing page, it's only really - https://analytics.ecosia.org/analytics.js - everything else seems to be static resources related to page content.

Running a search, the number of resources pulled in is refreshingly light - https://projectsstatic.bentasker.co.uk/MISC/MISC35/ecosia_search_requests.png - (can ignore how long the search took to run, currently battering my uplink with a backup).

One small disappointment, though:



Image Loading

if you look at the results page: https://projectsstatic.bentasker.co.uk/MISC/MISC35/ecosia_search_results.png on the left is a snippet from Wikipedia (which is fine), but the little thumbnail on it is loaded directly from Bing: https://www.bing.com/th?id=AMMS_27d93d5191f80594e1908e1b8dd7c8bc&w=110&h=110&c=7&rs=1&qlt=80&cdv=1&pid=16.1 as are all the thumbnails that you can see.

There are 4 Bing domains that we're loading content from in this query

- tse1.mm.bing.net (https://tse2.mm.bing.net/th?id=OVP.Fp3QvxQa0uMNpcZ57-AekAEsDh&pid=Api)
- tse2.mm.bing.net (https://tse2.mm.bing.net/th?id=OVP.Fp3QvxQa0uMNpcZ57-AekAEsDh&pid=Api)
- tse4.mm.bing.net (https://tse2.mm.bing.net/th?id=OVP.Fp3QvxQa0uMNpcZ57-AekAEsDh&pid=Api)
- www.bing.com (as above)

This gives Bing an opportunity to set cookies in order to track us (though they don't currently appear to be doing so).

Ecosia doesn't send a Referrer-Policy header, so by default we're sending Bing a referer header when loading these images. Ah, wait, they're using the meta-tag equivalent instead
<meta name="referrer" content="origin-when-crossorigin">

So, compliant browsers shouldn't leak search queries to Bing (still don't like that image requests go directly to them, but I suppose you've got to trust them at some point along the way as they're the underlying search provider)



Analytics

OK, so with adblock etc off, lets see what we get

The analytics code is loaded from here - https://analytics.ecosia.org/analytics.js. That results in this - https://ps.ecosia.org/FVh3ot2zPwDMi43LjI.js - being loaded. It's an instance of Snowplow: https://snowplowanalytics.com/
For companies who demand high-quality, real-time event data, delivered by a cloud-native data pipeline they fully control.

Sorry... "cloud-native data pipeline"... puke puke puke ... it's far too early in the day for that level of marketing bollocks.

As a result of that code running, we then make some POST requests to their analytics at sp.ecosio.org, both with JSON in the payload
{"schema":"iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4","data":[{"e":"ue","ue_px":"eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy91bnN0cnVjdF9ldmVudC9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6eyJzY2hlbWEiOiJpZ2x1Om9yZy5lY29zaWEvc2VhcmNoX2V2ZW50L2pzb25zY2hlbWEvMS0wLTUiLCJkYXRhIjp7InF1ZXJ5IjoiaGVsbG8gd29ybGQiLCJhZHNfc2hvd24iOiJhZF9kaXNwbGF5ZWQiLCJjb21wX3Nob3duIjoibm9fd2lkZ2V0IiwicGFnZV9udW0iOm51bGwsInNlYXJjaF90eXBlIjoic2VhcmNoIiwiZW50aXR5X3Nob3duIjp0cnVlLCJob3RlbF9zaG93biI6ZmFsc2UsImZsaWdodHNfc2hvd24iOmZhbHNlLCJtYXBfc25pcHBldF9zaG93biI6ZmFsc2UsImdyZWVuX2RvbWFpbnNfc2hvd24iOmZhbHNlfX19","tv":"js-2.7.2","tna":"sp_eco","p":"web","tz":"Europe/London","lang":"en-GB","cs":"UTF-8","f_pdf":"1","f_qt":"0","f_realp":"0","f_wma":"0","f_dir":"0","f_fla":"0","f_java":"0","f_gears":"0","f_ag":"0","res":"1920x1080","cd":"24","cookie":"1","eid":"5ba53fce-3d68-4629-9b06-56cb36018ec5","dtm":"1565684827012","cx":"eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6W3sic2NoZW1hIjoiaWdsdTpvcmcuZWNvc2lhL2VjZmdfY29udGV4dC9qc29uc2NoZW1hLzEtMC0xIiwiZGF0YSI6eyJsYW5ndWFnZSI6ImVuIiwiZGV2aWNldHlwZSI6InBjIiwidHJlZXMiOjgsIm1hcmtldCI6ImVuLWdiIiwiYXV0b3N1Z2dlc3QiOjEsImN1c3RvbSI6MCwibGFzdHRyZWUiOjE1NjU2ODQ4MTAsImZpcnN0c2VhcmNoIjowLCJmaXJzdHJ1biI6MCwiYWRkb24iOjAsInR5cGV0YWciOiJuYSIsInBlcnNvbmFsIjpudWxsLCJhZHVsdCI6ImkiLCJub3RpZmljYXRpb24iOjEsIm5ld3RhYiI6MCwiY291bnRyeSI6bnVsbCwiZWNmZ2Nvb2tpZSI6Im5mPTE6bHQ9MTU2NTY4NDgxMDphcz0xOnR1PWF1dG86bWM9ZW4tZ2I6ZHQ9cGM6bnQ9MDptYT0xOnR0PW5hOmw9ZW46ZnM9MDpmcj0wOnd1PWF1dG86Y3M9MDphPTA6Zj1pOnQ9OCJ9fSx7InNjaGVtYSI6ImlnbHU6b3JnLmVjb3NpYS9hZF9jb250ZXh0L2pzb25zY2hlbWEvMi0wLTEiLCJkYXRhIjp7InNpZGViYXIiOltdLCJtYWlubGluZSI6WyJ3d3cuYW1hem9uLmNvLnVrLyUzY2IlM2VIZWxsbyUyMFdvcmxkJTNjL2IlM2UiXSwiYm90dG9tIjpbInd3dy5hbWF6b24uY28udWsvJTNjYiUzZUhlbGxvJTIwV29ybGQlM2MvYiUzZSJdfX0seyJzY2hlbWEiOiJpZ2x1Om9yZy5lY29zaWEvc2xvdHRpbmdzX2NvbnRleHQvanNvbnNjaGVtYS8xLTAtMCIsImRhdGEiOnsic2xvdHRpbmdzIjpbIkFkcyIsIldlYlBhZ2VzIiwiVmlkZW9zIiwiUmVsYXRlZEZhY3RzIiwiUmVsYXRlZFNlYXJjaGVzIiwiRW50aXRpZXMiXX19LHsic2NoZW1hIjoiaWdsdTpvcmcuZWNvc2lhL2tub3dsZWRnZV9jb250ZXh0L2pzb25zY2hlbWEvMS0wLTAiLCJkYXRhIjp7IjAiOnsibmFtZSI6IiZxdW90O0hlbGxvLCBXb3JsZCEmcXVvdDsgcHJvZ3JhbSIsInR5cGVoaW50IjpbIkdlbmVyaWMiXSwiZW50aXR5X3BvcHVsYXRpb24iOlsiaWQiLCJuYW1lIiwiZGVzY3JpcHRpb24iLCJpbWFnZSIsImJpbmdJZCIsImVudGl0eVByZXNlbnRhdGlvbkluZm8iLCJ3ZWJTZWFyY2hVcmwiLCJ3ZWJTZWFyY2hVcmxQaW5nU3VmZml4IiwiY29udHJhY3R1YWxSdWxlcyIsInVybCJdfSwiMSI6eyJuYW1lIjoiSGVsbG9Xb3JsZCIsInR5cGVoaW50IjpbIk9yZ2FuaXphdGlvbiJdLCJlbnRpdHlfcG9wdWxhdGlvbiI6WyJpZCIsIm5hbWUiLCJkZXNjcmlwdGlvbiIsImltYWdlIiwiYmluZ0lkIiwiZW50aXR5UHJlc2VudGF0aW9uSW5mbyIsIndlYlNlYXJjaFVybCIsIndlYlNlYXJjaFVybFBpbmdTdWZmaXgiLCJjb250cmFjdHVhbFJ1bGVzIiwidXJsIl19fX1dfQ","vp":"1920x898","ds":"1905x2706","vid":"1","sid":"39d189a1-31dc-48cc-81bf-57966f555e76","duid":"81cb21f9-5407-4322-8d56-0e4c78665971","refr":"https://www.ecosia.org/search?q=hello+world+again","url":"https://www.ecosia.org/search?q=hello+world","stm":"1565684827016"}]}

and
{"schema":"iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4","data":[{"e":"se","se_ca":"entity","se_ac":"&quot;Hello, World!&quot; program","se_la":"Generic","se_pr":"hello world","tv":"js-2.7.2","tna":"sp_eco","p":"web","tz":"Europe/London","lang":"en-GB","cs":"UTF-8","f_pdf":"1","f_qt":"0","f_realp":"0","f_wma":"0","f_dir":"0","f_fla":"0","f_java":"0","f_gears":"0","f_ag":"0","res":"1920x1080","cd":"24","cookie":"1","eid":"8a3c0af0-db2f-46cb-822b-380c91a75cfd","dtm":"1565684827017","vp":"1920x898","ds":"1905x2706","vid":"1","sid":"39d189a1-31dc-48cc-81bf-57966f555e76","duid":"81cb21f9-5407-4322-8d56-0e4c78665971","refr":"https://www.ecosia.org/search?q=hello+world+again","url":"https://www.ecosia.org/search?q=hello+world","stm":"1565684827354"}]}


Let's uncrap those a bit. Looking at the first POST (https://projectsstatic.bentasker.co.uk/MISC/MISC35/post1.txt) , it starts off either uninteresting or self explanatory until we hit cx
{
    "data": [
        {
            "cd": "24",
            "cookie": "1",
            "cs": "UTF-8",
            "cx": "eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6W3sic2NoZW1hIjoiaWdsdTpvcmcuZWNvc2lhL2VjZmdfY29udGV4dC9qc29uc2NoZW1hLzEtMC0xIiwiZGF0YSI6eyJsYW5ndWFnZSI6ImVuIiwiZGV2aWNldHlwZSI6InBjIiwidHJlZXMiOjgsIm1hcmtldCI6ImVuLWdiIiwiYXV0b3N1Z2dlc3QiOjEsImN1c3RvbSI6MCwibGFzdHRyZWUiOjE1NjU2ODQ4MTAsImZpcnN0c2VhcmNoIjowLCJmaXJzdHJ1biI6MCwiYWRkb24iOjAsInR5cGV0YWciOiJuYSIsInBlcnNvbmFsIjpudWxsLCJhZHVsdCI6ImkiLCJub3RpZmljYXRpb24iOjEsIm5ld3RhYiI6MCwiY291bnRyeSI6bnVsbCwiZWNmZ2Nvb2tpZSI6Im5mPTE6bHQ9MTU2NTY4NDgxMDphcz0xOnR1PWF1dG86bWM9ZW4tZ2I6ZHQ9cGM6bnQ9MDptYT0xOnR0PW5hOmw9ZW46ZnM9MDpmcj0wOnd1PWF1dG86Y3M9MDphPTA6Zj1pOnQ9OCJ9fSx7InNjaGVtYSI6ImlnbHU6b3JnLmVjb3NpYS9hZF9jb250ZXh0L2pzb25zY2hlbWEvMi0wLTEiLCJkYXRhIjp7InNpZGViYXIiOltdLCJtYWlubGluZSI6WyJ3d3cuYW1hem9uLmNvLnVrLyUzY2IlM2VIZWxsbyUyMFdvcmxkJTNjL2IlM2UiXSwiYm90dG9tIjpbInd3dy5hbWF6b24uY28udWsvJTNjYiUzZUhlbGxvJTIwV29ybGQlM2MvYiUzZSJdfX0seyJzY2hlbWEiOiJpZ2x1Om9yZy5lY29zaWEvc2xvdHRpbmdzX2NvbnRleHQvanNvbnNjaGVtYS8xLTAtMCIsImRhdGEiOnsic2xvdHRpbmdzIjpbIkFkcyIsIldlYlBhZ2VzIiwiVmlkZW9zIiwiUmVsYXRlZEZhY3RzIiwiUmVsYXRlZFNlYXJjaGVzIiwiRW50aXRpZXMiXX19LHsic2NoZW1hIjoiaWdsdTpvcmcuZWNvc2lhL2tub3dsZWRnZV9jb250ZXh0L2pzb25zY2hlbWEvMS0wLTAiLCJkYXRhIjp7IjAiOnsibmFtZSI6IiZxdW90O0hlbGxvLCBXb3JsZCEmcXVvdDsgcHJvZ3JhbSIsInR5cGVoaW50IjpbIkdlbmVyaWMiXSwiZW50aXR5X3BvcHVsYXRpb24iOlsiaWQiLCJuYW1lIiwiZGVzY3JpcHRpb24iLCJpbWFnZSIsImJpbmdJZCIsImVudGl0eVByZXNlbnRhdGlvbkluZm8iLCJ3ZWJTZWFyY2hVcmwiLCJ3ZWJTZWFyY2hVcmxQaW5nU3VmZml4IiwiY29udHJhY3R1YWxSdWxlcyIsInVybCJdfSwiMSI6eyJuYW1lIjoiSGVsbG9Xb3JsZCIsInR5cGVoaW50IjpbIk9yZ2FuaXphdGlvbiJdLCJlbnRpdHlfcG9wdWxhdGlvbiI6WyJpZCIsIm5hbWUiLCJkZXNjcmlwdGlvbiIsImltYWdlIiwiYmluZ0lkIiwiZW50aXR5UHJlc2VudGF0aW9uSW5mbyIsIndlYlNlYXJjaFVybCIsIndlYlNlYXJjaFVybFBpbmdTdWZmaXgiLCJjb250cmFjdHVhbFJ1bGVzIiwidXJsIl19fX1dfQ",

That's actually a base64 encoded JSON string - https://projectsstatic.bentasker.co.uk/MISC/MISC35/cx_decoded.txt
{"schema":"iglu:com.snowplowanalytics.snowplow/contexts/jsonschema/1-0-0","data":[{"schema":"iglu:org.ecosia/ecfg_context/jsonschema/1-0-1","data":{"language":"en","devicetype":"pc","trees":8,"market":"en-gb","autosuggest":1,"custom":0,"lasttree":1565684810,"firstsearch":0,"firstrun":0,"addon":0,"typetag":"na","personal":null,"adult":"i","notification":1,"newtab":0,"country":null,"ecfgcookie":"nf=1:lt=1565684810:as=1:tu=auto:mc=en-gb:dt=pc:nt=0:ma=1:tt=na:l=en:fs=0:fr=0:wu=auto:cs=0:a=0:f=i:t=8"}},{"schema":"iglu:org.ecosia/ad_context/jsonschema/2-0-1","data":{"sidebar":[],"mainline":["www.amazon.co.uk/%3cb%3eHello%20World%3c/b%3e"],"bottom":["www.amazon.co.uk/%3cb%3eHello%20World%3c/b%3e"]}},{"schema":"iglu:org.ecosia/slottings_context/jsonschema/1-0-0","data":{"slottings":["Ads","WebPages","Videos","RelatedFacts","RelatedSearches","Entities"]}},{"schema":"iglu:org.ecosia/knowledge_context/jsonschema/1-0-0","data":{"0":{"name":"&quot;Hello, World!&quot; program","typehint":["Generic"],"entity_population":["id","name","description","image","bingId","entityPresentationInfo","webSearchUrl","webSearchUrlPingSuffix","contractualRules","url"]},"1":{"name":"HelloWorld","typehint":["Organization"],"entity_population":["id","name","description","image","bingId","entityPresentationInfo","webSearchUrl","webSearchUrlPingSuffix","contractualRules","url"]}}}]}

Curiously there are references to amazon.co.uk in there, but it's otherwise just standard fare for analytics. Does make you wonder why they b64'd it though rather than just having a nested object.

Anyway, moving on, the rest of the payload is mainly concered with things like

- Does my browser support flash
- Does my browser support java
- Does my browser support pdf
- Does my browser support wma

etc. Until we hit ue_px which is another (smaller) b64'd JSON string
            "ue_px": "eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy91bnN0cnVjdF9ldmVudC9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6eyJzY2hlbWEiOiJpZ2x1Om9yZy5lY29zaWEvc2VhcmNoX2V2ZW50L2pzb25zY2hlbWEvMS0wLTUiLCJkYXRhIjp7InF1ZXJ5IjoiaGVsbG8gd29ybGQiLCJhZHNfc2hvd24iOiJhZF9kaXNwbGF5ZWQiLCJjb21wX3Nob3duIjoibm9fd2lkZ2V0IiwicGFnZV9udW0iOm51bGwsInNlYXJjaF90eXBlIjoic2VhcmNoIiwiZW50aXR5X3Nob3duIjp0cnVlLCJob3RlbF9zaG93biI6ZmFsc2UsImZsaWdodHNfc2hvd24iOmZhbHNlLCJtYXBfc25pcHBldF9zaG93biI6ZmFsc2UsImdyZWVuX2RvbWFpbnNfc2hvd24iOmZhbHNlfX19",


This looks like it's essentially custom variables that Ecosia have added to show what's been included in the search results:
{
    "data": {
        "data": {
            "ads_shown": "ad_displayed",
            "comp_shown": "no_widget",
            "entity_shown": true,
            "flights_shown": false,
            "green_domains_shown": false,
            "hotel_shown": false,
            "map_snippet_shown": false,
            "page_num": null,
            "query": "hello world",
            "search_type": "search"
        },
        "schema": "iglu:org.ecosia/search_event/jsonschema/1-0-5"
    },
    "schema": "iglu:com.snowplowanalytics.snowplow/unstruct_event/jsonschema/1-0-0"
}


Ok, so looking at the second POST (https://projectsstatic.bentasker.co.uk/MISC/MISC35/post2.txt) it's almost duplicates the first, but contains more ID's, so I guess is linked to some kind of profile that Snowplow has now created for me.

So, they're collecting quite a lot of info about my browser, as well as how the results are formatted etc, but it's pretty standard analytics fare.



Ads

I don't actually see any ads on my test search. But then, assuming you want your ads to be relevant, what are you going to sell me when I search for "Hello World" (beginners programming books maybe?).

Running a new search for "Shoes" yields some ads - https://www.ecosia.org/search?q=shoes

They're actually fairly unobtrustive - https://projectsstatic.bentasker.co.uk/MISC/MISC35/shoes.png - the suggestion boxes at the top are themselves ads, going via Bing's aclick URL, so clicking one of those would generate Ecosia some revenue.

If I had one criticism it'd be that there are no organic results visible on my page in that screenshot (though, the ads do not extend over the fold, so it's a short scroll). That's OK if the ads return what I'm looking for (to be fair, they've offered up a reasonable range of different shoes there).

The ads are all image/text, no additional scripts have been loaded. So, actually, the ads still load with my adblockers enabled.

That's actually a little annoying, not because the ads are displayed, but because some of them don't work.

For example (I changed search terms here):

Taking the ad "Mens White Basic Vest" from Burtons
$ curl -o //dev/null -v "https://www.bing.com/aclick?ld=e3Xcfv6HDINa8D263ZHp15OTVUCUz5REFtMksSl74dxxWfCWrsvMTY2i3O4yBJ-YzyJ56H65zG1ubHhi0PoQUgacz8UqXVD3IN1_FZqp5qPjYGmM8TYdcAqj6kMSTX7clNLRWIPczWTyHtEQ4D92kTQ2uich0vctapHZIUZ1V6Hp8sjfR0&u=aHR0cCUzYSUyZiUyZmNsaWNrc2VydmUuZGFydHNlYXJjaC5uZXQlMmZsaW5rJTJmY2xpY2slM2ZsaWQlM2Q5MjcwMDA0NDMwMDQwMDAzNyUyNmRzX3Nfa3dnaWQlM2Q1ODcwMDAwMTk4NjA4MDU2OCUyNmRzX3NfaW52ZW50b3J5X2ZlZWRfaWQlM2Q5NzcwMDAwMDAwMTAwMDQ4MCUyNiUyNmRzX2VfYWRpZCUzZDc3MTcxOTc2MzY0MTQ0JTI2ZHNfZV9wcm9kdWN0X2dyb3VwX2lkJTNkNDU4MDc3MTYwNTgxMjE4NCUyNmRzX2VfcHJvZHVjdF9pZCUzZDE1MjAxOTAwMDQ0MTI5NCUyNmRzX2VfcHJvZHVjdF9jb3VudHJ5JTNkR0IlMjZkc19lX3Byb2R1Y3RfbGFuZ3VhZ2UlM2RFTiUyNmRzX2VfcHJvZHVjdF9jaGFubmVsJTNkb25saW5lJTI2ZHNfdXJsX3YlM2QyJTI2ZHNfZGVzdF91cmwlM2RodHRwcyUzYSUyZiUyZnd3dy5idXJ0b24uY28udWslMmZlbiUyZmJydWslMmZwcm9kdWN0JTJmY2xvdGhpbmctMjgxNTU5JTJmbWVucy10LXNoaXJ0cy12ZXN0cy0yNjcyODM4JTJmd2hpdGUtYmFzaWMtdmVzdC04ODQzMDU0JTNmZ2VvaXAlM2Rub3JlZGlyZWN0JTI2Y21waWQlM2RwcGNfcGxhX1VLX2lwJTI2aXN0Q29tcGFueUlkJTNkMmNlOTZiYTItZGNkNS00MTZlLTlkMmUtNDE5YmVhYWU3MmIwJTI2aXN0RmVlZElkJTNkMmQ1OTcxNTUtZWI5Yy00ZDgzLWE2ZTQtODU3NjUyMmYzOTU0JTI2aXN0SXRlbUlkJTNkd2FseHJ4cWl3JTI2aXN0QmlkJTNkdHp0eCUyNm1zY2xraWQlM2RjNWM1ZTYyY2NhOWIxNjA3NzU5OGJiN2ViMGIxYzU1NiUyNnV0bV9zb3VyY2UlM2RiaW5nJTI2dXRtX21lZGl1bSUzZGNwYyUyNnV0bV9jYW1wYWlnbiUzZFBMQSUyNTIwLSUyNTIwQlMlMjUyMC0lMjUyMChCZXN0JTI1MjBTZWxsZXJzKSUyNTIwLSUyNTIwSElHSCUyNnV0bV90ZXJtJTNkNDU4MDc3MTYwNTgxMjE4NCUyNnV0bV9jb250ZW50JTNkUExBJTI1MjAtJTI1MjBCUyUyNTIwLSUyNTIwKEJlc3QlMjUyMFNlbGxlcnMpJTI1MjAtJTI1MjBISUdI&rlid=c5c5e62cca9b16077598bb7eb0b1c556"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0*   Trying 204.79.197.200...
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0* Connected to www.bing.com (204.79.197.200) port 443 (#0)
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 603 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* 	 server certificate verification OK
* 	 server certificate status verification SKIPPED
* 	 common name: www.bing.com (matched)
* 	 server certificate expiration date OK
* 	 server certificate activation date OK
* 	 certificate public key: RSA
* 	 certificate version: #3
* 	 subject: CN=www.bing.com
* 	 start date: Tue, 30 Apr 2019 20:48:00 GMT
* 	 expire date: Fri, 30 Apr 2021 20:48:00 GMT
* 	 issuer: C=US,ST=Washington,L=Redmond,O=Microsoft Corporation,OU=Microsoft IT,CN=Microsoft IT TLS CA 2
* 	 compression: NULL
* ALPN, server accepted to use http/1.1
> GET /aclick?ld=e3Xcfv6HDINa8D263ZHp15OTVUCUz5REFtMksSl74dxxWfCWrsvMTY2i3O4yBJ-YzyJ56H65zG1ubHhi0PoQUgacz8UqXVD3IN1_FZqp5qPjYGmM8TYdcAqj6kMSTX7clNLRWIPczWTyHtEQ4D92kTQ2uich0vctapHZIUZ1V6Hp8sjfR0&u=aHR0cCUzYSUyZiUyZmNsaWNrc2VydmUuZGFydHNlYXJjaC5uZXQlMmZsaW5rJTJmY2xpY2slM2ZsaWQlM2Q5MjcwMDA0NDMwMDQwMDAzNyUyNmRzX3Nfa3dnaWQlM2Q1ODcwMDAwMTk4NjA4MDU2OCUyNmRzX3NfaW52ZW50b3J5X2ZlZWRfaWQlM2Q5NzcwMDAwMDAwMTAwMDQ4MCUyNiUyNmRzX2VfYWRpZCUzZDc3MTcxOTc2MzY0MTQ0JTI2ZHNfZV9wcm9kdWN0X2dyb3VwX2lkJTNkNDU4MDc3MTYwNTgxMjE4NCUyNmRzX2VfcHJvZHVjdF9pZCUzZDE1MjAxOTAwMDQ0MTI5NCUyNmRzX2VfcHJvZHVjdF9jb3VudHJ5JTNkR0IlMjZkc19lX3Byb2R1Y3RfbGFuZ3VhZ2UlM2RFTiUyNmRzX2VfcHJvZHVjdF9jaGFubmVsJTNkb25saW5lJTI2ZHNfdXJsX3YlM2QyJTI2ZHNfZGVzdF91cmwlM2RodHRwcyUzYSUyZiUyZnd3dy5idXJ0b24uY28udWslMmZlbiUyZmJydWslMmZwcm9kdWN0JTJmY2xvdGhpbmctMjgxNTU5JTJmbWVucy10LXNoaXJ0cy12ZXN0cy0yNjcyODM4JTJmd2hpdGUtYmFzaWMtdmVzdC04ODQzMDU0JTNmZ2VvaXAlM2Rub3JlZGlyZWN0JTI2Y21waWQlM2RwcGNfcGxhX1VLX2lwJTI2aXN0Q29tcGFueUlkJTNkMmNlOTZiYTItZGNkNS00MTZlLTlkMmUtNDE5YmVhYWU3MmIwJTI2aXN0RmVlZElkJTNkMmQ1OTcxNTUtZWI5Yy00ZDgzLWE2ZTQtODU3NjUyMmYzOTU0JTI2aXN0SXRlbUlkJTNkd2FseHJ4cWl3JTI2aXN0QmlkJTNkdHp0eCUyNm1zY2xraWQlM2RjNWM1ZTYyY2NhOWIxNjA3NzU5OGJiN2ViMGIxYzU1NiUyNnV0bV9zb3VyY2UlM2RiaW5nJTI2dXRtX21lZGl1bSUzZGNwYyUyNnV0bV9jYW1wYWlnbiUzZFBMQSUyNTIwLSUyNTIwQlMlMjUyMC0lMjUyMChCZXN0JTI1MjBTZWxsZXJzKSUyNTIwLSUyNTIwSElHSCUyNnV0bV90ZXJtJTNkNDU4MDc3MTYwNTgxMjE4NCUyNnV0bV9jb250ZW50JTNkUExBJTI1MjAtJTI1MjBCUyUyNTIwLSUyNTIwKEJlc3QlMjUyMFNlbGxlcnMpJTI1MjAtJTI1MjBISUdI&rlid=c5c5e62cca9b16077598bb7eb0b1c556 HTTP/1.1
> Host: www.bing.com
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 302 Object Moved
< Cache-Control: no-store
< Pragma: no-cache
< Transfer-Encoding: chunked
< Expires: Fri, 01 Jan 1990 00:00:00 GMT
< Location: http://clickserve.dartsearch.net/link/click?lid=92700044300400037&ds_s_kwgid=58700001986080568&ds_s_inventory_feed_id=97700000001000480&&ds_e_adid=77171976364144&ds_e_product_group_id=4580771605812184&ds_e_product_id=152019000441294&ds_e_product_country=GB&ds_e_product_language=EN&ds_e_product_channel=online&ds_url_v=2&ds_dest_url=https://www.burton.co.uk/en/bruk/product/clothing-281559/mens-t-shirts-vests-2672838/white-basic-vest-8843054?geoip=noredirect&cmpid=ppc_pla_UK_ip&istCompanyId=2ce96ba2-dcd5-416e-9d2e-419beaae72b0&istFeedId=2d597155-eb9c-4d83-a6e4-8576522f3954&istItemId=walxrxqiw&istBid=tztx&msclkid=c5c5e62cca9b16077598bb7eb0b1c556&utm_source=bing&utm_medium=cpc&utm_campaign=PLA%20-%20BS%20-%20(Best%20Sellers)%20-%20HIGH&utm_term=4580771605812184&utm_content=PLA%20-%20BS%20-%20(Best%20Sellers)%20-%20HIGH
< P3P: CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"
< P3P: CP=BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo
< Referrer-Policy: origin
< X-MSEdge-Ref: Ref A: 3E80D8EC4651433C949037306F7A62FC Ref B: LON04EDGE0221 Ref C: 2019-08-13T08:53:04Z
< Set-Cookie: _EDGE_S=F=1&SID=23A29C47BBD46034257591F0BA94619A; path=/; httponly; domain=bing.com
< Set-Cookie: _EDGE_V=1; path=/; httponly; expires=Sun, 06-Sep-2020 08:53:04 GMT; domain=bing.com
< Set-Cookie: MUID=176FAC8B89836B0033C4A13C88C36A09; path=/; expires=Sun, 06-Sep-2020 08:53:04 GMT; domain=bing.com
< Set-Cookie: MUIDB=176FAC8B89836B0033C4A13C88C36A09; path=/; httponly; expires=Sun, 06-Sep-2020 08:53:04 GMT
< Date: Tue, 13 Aug 2019 08:53:04 GMT
< 
{ [975 bytes data]
100   968    0   968    0     0    273      0 --:--:--  0:00:03 --:--:--   273
* Connection #0 to host www.bing.com left intact

That redirects onto another tracker - clickserve.dartsearch.net - which is blocked, so I can't access the content.


Other adverts though, are better behaved. There's an Ad from Matalan for a "Mens US Athletic Side Stripe Vest", and they've set Bing to redirect straight to them
ben@milleniumfalcon:/tmp/MISC_35_Ecosia$ curl -s -v -o/dev/null "https://www.bing.com/aclick?ld=e3B1hNE7-Kl8ckEn78ULh47jVUCUzNS4xtuOqSasgRRvJUd5A2-y-9SB1U41yB_9zz5ajCFnKHIVgvsIMG9p6xbZH03lYIDSy-hRZSOD7CTaagHD6pn68O-qPKLv7_KMyio_hee9BG_7TPUFuEkpkRlCrdHNk5nj5Fb-UrxUFQJQbTEIaM&u=aHR0cCUzYSUyZiUyZnd3dy5tYXRhbGFuLmNvLnVrJTJmcHJvZHVjdCUyZmRldGFpbCUyZnMyNzI2NDMyX2MyNzAlMmZ1cy1hdGhsZXRpYy1zaWRlLXN0cmlwZS12ZXN0LW5hdnklM2Ztc2Nsa2lkJTNkNmY4ZTY3N2UyNjc1MWE2NmZlNzEzYTJjNWM1ZjM0NjglMjZ1dG1fc291cmNlJTNkYmluZyUyNnV0bV9tZWRpdW0lM2RjcGMlMjZ1dG1fY2FtcGFpZ24lM2RTaG9wcGluZyUyNTIwLSUyNTIwQ2F0Y2glMjUyMEFsbCUyNnV0bV90ZXJtJTNkNDU4MDQ5NjcyODEyNTcxMSUyNnV0bV9jb250ZW50JTNkQ2F0Y2glMjUyMEFsbA&rlid=6f8e677e26751a66fe713a2c5c5f3468"
*   Trying 13.107.21.200...
* Connected to www.bing.com (13.107.21.200) port 443 (#0)
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 603 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
* 	 server certificate verification OK
* 	 server certificate status verification SKIPPED
* 	 common name: www.bing.com (matched)
* 	 server certificate expiration date OK
* 	 server certificate activation date OK
* 	 certificate public key: RSA
* 	 certificate version: #3
* 	 subject: CN=www.bing.com
* 	 start date: Tue, 30 Apr 2019 20:48:00 GMT
* 	 expire date: Fri, 30 Apr 2021 20:48:00 GMT
* 	 issuer: C=US,ST=Washington,L=Redmond,O=Microsoft Corporation,OU=Microsoft IT,CN=Microsoft IT TLS CA 2
* 	 compression: NULL
* ALPN, server accepted to use http/1.1
> GET /aclick?ld=e3B1hNE7-Kl8ckEn78ULh47jVUCUzNS4xtuOqSasgRRvJUd5A2-y-9SB1U41yB_9zz5ajCFnKHIVgvsIMG9p6xbZH03lYIDSy-hRZSOD7CTaagHD6pn68O-qPKLv7_KMyio_hee9BG_7TPUFuEkpkRlCrdHNk5nj5Fb-UrxUFQJQbTEIaM&u=aHR0cCUzYSUyZiUyZnd3dy5tYXRhbGFuLmNvLnVrJTJmcHJvZHVjdCUyZmRldGFpbCUyZnMyNzI2NDMyX2MyNzAlMmZ1cy1hdGhsZXRpYy1zaWRlLXN0cmlwZS12ZXN0LW5hdnklM2Ztc2Nsa2lkJTNkNmY4ZTY3N2UyNjc1MWE2NmZlNzEzYTJjNWM1ZjM0NjglMjZ1dG1fc291cmNlJTNkYmluZyUyNnV0bV9tZWRpdW0lM2RjcGMlMjZ1dG1fY2FtcGFpZ24lM2RTaG9wcGluZyUyNTIwLSUyNTIwQ2F0Y2glMjUyMEFsbCUyNnV0bV90ZXJtJTNkNDU4MDQ5NjcyODEyNTcxMSUyNnV0bV9jb250ZW50JTNkQ2F0Y2glMjUyMEFsbA&rlid=6f8e677e26751a66fe713a2c5c5f3468 HTTP/1.1
> Host: www.bing.com
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 302 Object Moved
< Cache-Control: no-store
< Pragma: no-cache
< Transfer-Encoding: chunked
< Expires: Fri, 01 Jan 1990 00:00:00 GMT
< Location: http://www.matalan.co.uk/product/detail/s2726432_c270/us-athletic-side-stripe-vest-navy?msclkid=6f8e677e26751a66fe713a2c5c5f3468&utm_source=bing&utm_medium=cpc&utm_campaign=Shopping%20-%20Catch%20All&utm_term=4580496728125711&utm_content=Catch%20All
< P3P: CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"
< P3P: CP=BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo
< Referrer-Policy: origin
< X-MSEdge-Ref: Ref A: 87274BB3CDB14EFCA60007F0940CF2B8 Ref B: LON04EDGE0408 Ref C: 2019-08-13T08:55:11Z
< Set-Cookie: _EDGE_S=F=1&SID=1B31E606F6166FF32A75EBB1F7566ED2; path=/; httponly; domain=bing.com
< Set-Cookie: _EDGE_V=1; path=/; httponly; expires=Sun, 06-Sep-2020 08:55:11 GMT; domain=bing.com
< Set-Cookie: MUID=0A7FFDA5A4776E6B0FC7F012A5376F89; path=/; expires=Sun, 06-Sep-2020 08:55:11 GMT; domain=bing.com
< Set-Cookie: MUIDB=0A7FFDA5A4776E6B0FC7F012A5376F89; path=/; httponly; expires=Sun, 06-Sep-2020 08:55:11 GMT
< Date: Tue, 13 Aug 2019 08:55:10 GMT
< 
{ [399 bytes data]

Which means I can see and buy their product.

It's not really something that Ecosia can do much about themselves, but it may get a little annoying if I'm regularly finding "results" won't work.

On the other hand, I guess it's a self-selecting pool: those who insist on associating with the likes of dartsearch (like Burton's above) will get no custom (but presumably still have to pay Bing for clicks, allowing Ecosia to plant more trees), while those (like Matalan above) who do not take the piss can still potentially get custom.

I guess I'll just wait and see how I get on with it.

What I am going to do, though, is

- Not whitelist Ecosia's analytics (i.e continue blocking it)
- Tell Privacy Badger that it should prevent cookies from being sent to Bing's domains (when they're a 3rd party)

Repo: adblocklists
Commit: 1668e90b31a11a277940d7b2d646e38b08ccc582
Author: B Tasker <github@<Domain Hidden>>

Date: Tue Aug 13 10:07:13 2019 +0100
Commit Message: Block Ecosia's analytics (identified in MISC-35)



Modified (-)(+)
-------
config/manualblock.txt
config/manualpages.txt




Webhook User-Agent

GitHub-Hookshot/d410879


View Commit

Worth noting though, I have made the effort this morning to make sure every search I've done while looking into this (and some other stuff) has been via Ecosia, and it's not actually let me down yet. I guess Bing must've made some improvements since I last used it.
Figured I may as well make Ecosia the default in on my desktop too



Chromium

Settings -> Manage Search Engines -> Add

- Search Engine: Ecosia
- Keyword: Ecosia
- URL: https://www.ecosia.org/search?q=%s

Like this: https://projectsstatic.bentasker.co.uk/MISC/MISC35/adding_ecosia_to_chromium.png

Then find it in the list, click the 3 dots and choose Make default



Firefox

Install the Ecosia addon - https://addons.mozilla.org/en-US/firefox/addon/ecosia-the-green-search/?src=search
Seems to be working OK for me so far, so I've said sod it and embedded it into projects.bentasker.co.uk:
function embedSearch(){

    var logo = document.getElementById('logoarea');
    var cont = logo.parentNode;
    
    var d = document.createElement('div');
    d.id = 'searcharea';
    
    var f = document.createElement('form');
    f.addEventListener('submit',triggerSearch);

    var i = document.createElement('input');
    i.setAttribute('type','text');
    i.setAttribute('placeholder','search');
    i.setAttribute('class','searchbox');
    i.setAttribute('id','searchstring');
    i.addEventListener('keypress',setupPreConnect);
    
    f.appendChild(i);
    
    var b = document.createElement('button');
    b.addEventListener('click',triggerSearch);
    b.innerHTML='Search';
    
    f.appendChild(b);
    d.appendChild(f);
    cont.insertBefore(d,logo);
    
    s = document.createElement('style');
    s.setAttribute('type','text/css');
    s.innerHTML = '#searcharea {float: right;} #searchbox {margin-top: 10px;}'
    
    document.getElementsByTagName('head')[0].appendChild(s);
    
}

function setupPreConnect(){
    // https://snippets.bentasker.co.uk/page-1908140902-Make-the-browser-preconnect-to-a-domain-via-javascript-Javascript.html

    // We wrap in a conditional so we're not spamming the DOM with every keypress
    if (!window.ecosiapreconnected){

        // The user has started typing in the text box, so take that as a sign we're going to be going to Ecosia.
        // Tell the browser to pre-connect to speed up the response
        var l = document.createElement('link');
        l.rel = 'preconnect';
        l.href = "https://www.ecosia.org";
        document.head.appendChild(l);
        window.ecosiapreconnected = 1;
    }
}

function triggerSearch(e){
    var searchstr = document.getElementById("searchstring").value;
    var qs = encodeURI("site:" + window.location.hostname + " " + searchstr);
    console.log("Generating search for " + qs);

    window.location.href = 'https://www.ecosia.org/search?q=' + qs;

    e.preventDefault();
}


There do seem to be a few issues missing from Bing's index (GPXIN-1 being one that doesn't show up in Ecosia, but is the first hit in Google).

I suspect, though, that's more to do with my habit of ignoring optimising for Bing than anything. That does tie in with my concerns about cutting over though - a lot of webmaster's ignore Bing so there's a high chance I may find it just hasn't indexed the content _I_ need some of the time.
I never thought I'd say this, but Bing's Webmaster Tools is now far, far superior to Google's Webmaster Tools.

In fairness, that's partly because Google have knackered their own product by switching to a new UI which just isn't a good fit for the use I make of it, but either way, the information is so so much more accessible in Bing's offering.
No issues so far, I've gradually been switching devices over to Ecosia, and haven't yet had to fall back to Google (though I think Ecosia has a search with google function - not tried it).
Still no real issues, and I've got Ecosia set as the default everywhere now.

I have noticed a very slight change in habit for some things though. Occasionally I'll be searching for something within a given site (rather than browsing to that site first... lazy). If the page doesn't come up near the top of the results, I'll tend to click into the first result from that site and then use the site's internal search.

A good example of this in practice is probably TheTVDB - got to find an affected series now...

Ok, search term "thetvdb Bamboo Blade".

At time of writing:

- Ecosia doesn't return a result pointing to the series record on TheTVDB.com - https://www.ecosia.org/search?q=thetvdb+Bamboo+blade
- TheTVDB has it, and it can be found with the internal search - https://www.thetvdb.com/search?q=Bamboo+Blade&l=en
- Google returns it at the top of page 1- https://www.google.com/search?q=thetvdb+bamboo+blade

Actually, looking closer at the Ecosia results, it is there it's just at the bottom of page 1.

So, at a technical and usability level, Ecosia did a worse job of returning the desired result than Google (quite why it ordered wikipedia, youtube, themoviedb, tuanimeonline, fanart.tv and funimation.com before a result with all the search terms in the URL as well as page content...).

Obviously, Ecosia/Bing as search operators so searching site:thetvdb.com Bamboo blade avoids that.

The underlying concern in all of this though, is what the impact is when I'm searching for something where I don't know the domain it'll be on. Based on the last few weeks though, the answer seems to be that you won't be aware of it, and so won't notice. I haven't failed to find anything I've had to search for (including technical stuff), and I haven't felt like I've had to spend ages searching for something.

I'm going to close this issue off now, and keep Ecosia as my SE as although it's possible to find specific examples where Google performs better, I've not noticed sufficient real-life impact from it to be even inclined to switch back.

btasker changed status from 'Open' to 'Resolved'
btasker added 'Done' to resolution
btasker changed status from 'Resolved' to 'Closed'
I've written up the experience here - https://www.bentasker.co.uk/blog/privacy/479-breaking-the-google-addiction-one-step-at-a-time - hopefully down the line, this ticket will have others alongside it successfully moving me off other Google services.
-------------------------
From: Ben Tasker <jira@<Domain Hidden>>
To: "Ben Tasker (JIRA)" <jiraacct@<Domain Hidden>>
Date: Wed, 26 Aug 2020 22:09:33 +0100
Subject: MISC-35
-------------------------


The new Firefox for Android has landed on my phone, and has wiped the
original settings and set my default search back to Google

Process to add Ecosia on the new version is less simple than the old
version:
Settings -> Search -> Add

Name: Ecosia
Search String: https://www.ecosia.org/search?q=%s


Doesn't seem to have the ability to hold down on a search box and simply
choose "Add search" anymore