At some point in the near future, I'm intending to make changes to my search infrastructure.
One of the things I want to do, is to make the search bot a little smarter in terms of what's important within a page, and what isn't.
In order to aid with that, I want it to process microdata (being implemented in
JILS-17), as well as pulling alternative representations where they are available.
Initially this will essentially be the sort of output you'd expect from a REST API. Once the framework to generate this is in place, it may later expand to add support for JSON-LD
Activity
2016-04-24 18:23:59
Queries need to be separated from page rendering so that the data can be extracted from the database, and then returned in the appropriate format.
2016-04-24 18:25:05
2016-04-24 18:25:56
2016-04-24 19:09:26
Webhook User-Agent
View Commit
2016-04-24 19:17:06
Currently the structure is
I'm going to start by tweaking to essentially be
2016-04-24 19:29:38
One thing that needs thinking about, is how the rewrites are going to work when using a mirror script. If that representation were dumped onto projects.bentasker.co.uk, the leading slash would lead to 404's (as the mirror lives in a subdirectory). So, we probably want to take X-SITEMAP-BASE into account
2016-04-24 19:33:58
2016-04-24 19:35:26
Webhook User-Agent
View Commit
2016-04-24 19:35:27
Webhook User-Agent
View Commit
2016-04-24 19:35:28
Webhook User-Agent
View Commit
2016-04-24 19:35:55
2016-04-24 20:01:28
Webhook User-Agent
View Commit
2016-04-24 20:04:09
Need to populate with versions/components
2016-04-24 20:05:28
2016-04-24 20:22:42
2016-04-24 20:39:22
2016-04-24 21:01:27
Webhook User-Agent
View Commit
2016-04-24 21:01:27
Webhook User-Agent
View Commit
2016-04-24 21:03:05
2016-04-24 21:03:17
2016-04-24 21:03:35
2016-04-24 21:27:29
2016-04-24 21:29:26
Webhook User-Agent
View Commit
2016-04-24 21:32:21
2016-04-24 22:01:27
Webhook User-Agent
View Commit
2016-04-24 23:21:26
Webhook User-Agent
View Commit
2016-04-24 23:21:26
Webhook User-Agent
View Commit
2016-04-24 23:21:29
Basic testing has been done, though need to look into creating a client script that attempts to walk the JSON to ensure that it's navigable.
2016-04-24 23:35:19
Other than that, I think we're done.
2016-04-24 23:35:26
Webhook User-Agent
View Commit
2016-04-24 23:36:13
2016-04-24 23:36:24
2016-04-24 23:36:24
2016-04-24 23:36:28
2016-04-29 14:54:03
2016-04-29 14:54:03
2016-04-29 14:54:03
2016-04-29 14:55:26
2016-04-29 14:55:26
2016-04-29 14:55:26
2016-04-29 14:55:26
2016-04-29 15:01:31
2016-04-29 15:01:31
2016-04-29 15:01:38
2016-04-29 15:01:38
2016-04-29 15:02:18
2016-04-29 15:02:18
2016-04-29 15:02:18
2016-04-29 15:02:22