project Websites / Gitlab Issue Listing Script avatar

websites/Gitlab-Issue-Listing-Script#12: HTTP 500s returned



Issue Information

Issue Type: issue
Status: closed
Reported By: btasker
Assigned To: btasker

Milestone: v0.3
Created: 21-Oct-21 15:29



Description

When my search spider tries to hit

  • https://gils.bentasker.co.uk/project/sysconfigs/spider_config.html
  • https://gils.bentasker.co.uk/commits/_nginx_config/5f4c0a64105f7a665778793b6902dfffb6353beb.html

It gets a HTTP 500.

It's most likely to be the result of an authentication issue - the GILS user doesn't have access to sysconfigs but we should handle this more cleanly.



Toggle State Changes

Activity


assigned to @btasker

Running in a dev instance with error display turned back on, we get

<b>Fatal error</b>:  Uncaught TypeError: count(): Argument [[#1](/issue/websites/Gitlab-Issue-Listing-Script/1.html)](/issue/websites/Gitlab-Issue-Listing-Script/1.html) ($value) must be of type Countable|array, stdClass given in /var/www/html/lib/gitlab.php:226
Stack trace:
[#0](/issue/websites/Gitlab-Issue-Listing-Script/0.html) /var/www/html/lib/router.php(58): GILSGitlab-&gt;getProjectAndIssues('sysconfigs/spid...')
[[#1](/issue/websites/Gitlab-Issue-Listing-Script/1.html)](/issue/websites/Gitlab-Issue-Listing-Script/1.html) /var/www/html/index.php(12): GILSRouter-&gt;route('/project/syscon...')
[#2](/issue/websites/Gitlab-Issue-Listing-Script/2.html) {main}
  thrown in <b>/var/www/html/lib/gitlab.php</b> on line <b>226</b><br>

So, it turns out to be a weird combination of permissions.

The GILS user has permission to view the project, but not the container registry.

The call that's failing is to list containers, rather than returning an array of containers we return an object (containing the error message).

The correct way to handle this, then is to catch that error.

verified

mentioned in commit ecfbe30bdc4b461f9ac8637b06b6f6165a498942

Commit: ecfbe30bdc4b461f9ac8637b06b6f6165a498942 
Author: B Tasker                            
                            
Date: 2021-10-21T18:02:27.000+01:00 

Message

Correctly catch when we're denied access to a container registry. See websites/Gitlab-Issue-Listing-Script#12

+4 -0 (4 lines changed)

Actually, looking at the logs, we also get for https://gils.bentasker.co.uk/commits/_nginx_config/5f4c0a64105f7a665778793b6902dfffb6353beb.html

Updating the description

changed the description

This second one is

Warning</b>:  Undefined property: stdClass::$title in <b>/var/www/html/lib/gitlab.php</b> on line <b>22</b><br />
<br />
<b>Warning</b>:  Attempt to read property "note" on string in <b>/var/www/html/lib/gitlab.php</b> on line <b>36</b><br />
<br />
<b>Fatal error</b>:  Uncaught Error: Attempt to assign property &quot;note&quot; on string in /var/www/html/lib/gitlab.php:36
Stack trace:
[#0](/issue/websites/Gitlab-Issue-Listing-Script/0.html) /var/www/html/lib/router.php(41): GILSGitlab-&gt;getCommit('_nginx_config/5...', '5f4c0a64105f7a6...')
[#1](/issue/websites/Gitlab-Issue-Listing-Script/1.html) /var/www/html/index.php(12): GILSRouter-&gt;route('/commits/_nginx...')
[#2](/issue/websites/Gitlab-Issue-Listing-Script/2.html) {main}
  thrown in <b>/var/www/html/lib/gitlab.php</b> on line <b>36</b><br />

It turns out GL will return a slightly different string if it's the project that doesn't exist.

404 Commit Not Found

vs

404 Project Not Found

That we're seeing this suggests there may be a typo somewhere - it's possible there's a bug somewhere else, or it might be that this has been incorrectly linked to from within an issue. Worth checking, but we're definitely missing group name there, and the repo name also looks fairly suspect

verified

mentioned in commit 7b28d876e10d7aec0fb032395ea02cd12c3b1030

Commit: 7b28d876e10d7aec0fb032395ea02cd12c3b1030 
Author: B Tasker                            
                            
Date: 2021-10-21T18:10:20.000+01:00 

Message

Fix 404 handling on commits - don't use direct string matching websites/Gitlab-Issue-Listing-Script#12

It turns out GL will return a slightly different string if it's the project that doesn't exist.

That we're seeing this suggests there may be a typo somewhere - it's possible there's a bug somewhere else, or it might be that this has been incorrectly linked to from within an issue. Worth checking

+2 -2 (4 lines changed)

There's no reference in last night's backup

ben@holly:/mnt/work/gilshtml$ grep -Rl "_nginx_config/5f4c0a64105f7a665778793b6902dfffb6353beb" *
ben@holly:/mnt/work/gilshtml$ 

So it was probably introduced in the latest tag

Running a new mirror

mkdir /mnt/scratch/tmpgils
cd /mnt/scratch/tmpgils

JIRALIST="https://gils.bentasker.co.uk" # Set your JIRA url here
wget --no-check-certificate -A "$PROJECT*" -R "robots.txt" -U "Jira-Project-Archive" -r -p -k "$JIRALIST/"

Checking in that

ben@holly:/mnt/scratch/tmpgils$ grep -Rl "_nginx_config/5f4c0a64105f7a665778793b6902dfffb6353beb" *
gils.bentasker.co.uk/issue/jira-projects/LAN/5.html

ben@holly:/mnt/scratch/tmpgils$ grep -R "_nginx_config/5f4c0a64105f7a665778793b6902dfffb6353beb" *
gils.bentasker.co.uk/issue/jira-projects/LAN/5.html:                    <p>mentioned in commit sysconfigs/pihrp1<a href="https://gils.bentasker.co.uk/commits/_nginx_config/5f4c0a64105f7a665778793b6902dfffb6353beb.html">5f4c0a6</a></p>

Looks like a misfire in GILSUtils::replaceCommitRefsEtc() then

Looks like this is still an issue Screenshot_20220418_105356 (from path /issue/jira-projects/LAN/5.html)

In Gitlab it looks like this Screenshot_20220418_105520

Must be something wrong with the regex, but I've got admit I don't immediately see it

It's because we don't allow numbers in project names Screenshot_20220418_105917

Adding 0-9 to the pattern has fixed that, but curiously the generated commit link 404s, so we may yet have another issue.

Trying to view any commits (/commits/sysconfigs/pihrp1_nginx_config/list.html) in that project fails, so I'm guessing this is probably permissions on Gitlab's side.

It's not permissions.

When the API call was introduced in 6c87c56e the invocation was added as

$r = $this->callAPI("GET", $url, false, false);

The result is that authentication is turned off (that final false) for the call - so we can't access the project because it's not public.

This has some overlap with #15 as that was concerned with the fact that we were only enforcing the auth toggle for the projects list.

I'll fix this under that issue - two birds, one stone etc

mentioned in issue #15

verified

mentioned in commit 2dba8a9c8c50f0c3c69f0b1d447fc485e6aef7df

Commit: 2dba8a9c8c50f0c3c69f0b1d447fc485e6aef7df 
Author: B Tasker                            
                            
Date: 2022-04-18T11:21:22.000+01:00 

Message

Allow numbers in project names for websites/Gitlab-Issue-Listing-Script#12

+1 -1 (2 lines changed)

This is fixed.

There is, however one more thing to fix - we need to make sure that a 404 is correctly returned if a commit can't be retrieved (whether because of auth or whatever).

verified

mentioned in commit 60fbdeb10138ac958f3d18283a5d428af6e248c5

Commit: 60fbdeb10138ac958f3d18283a5d428af6e248c5 
Author: B Tasker                            
                            
Date: 2022-04-18T11:33:20.000+01:00 

Message

Generate a 404 if a project does not exist when displaying a commit list. See websites/Gitlab-Issue-Listing-Script#12

+7 -1 (8 lines changed)