project Websites / Gitlab Issue Listing Script avatar

websites/Gitlab-Issue-Listing-Script#19: Only public projects are being listed despite public_projects_only being disabled



Issue Information

Issue Type: issue
Status: closed
Reported By: btasker
Assigned To: btasker

Milestone: v0.3
Created: 16-Apr-22 12:57



Description

I noticed the homepage was quite short.

I had public_projects_only turned on in config, but turning it off made no difference.



Toggle State Changes

Activity


assigned to @btasker

verified

mentioned in commit c88f2a3c1b9613da3d2e0253f1e6425dac5d0b82

Commit: c88f2a3c1b9613da3d2e0253f1e6425dac5d0b82 
Author: B Tasker                            
                            
Date: 2022-04-17T09:05:55.000+01:00 

Message

Fix issue with only some projects being displayed on project listing page (websites/Gitlab-Issue-Listing-Script#19)

I initially thought it was an issue with the auth toggle, but it's actually that the API someimes returns a leading space in front of a URL, which PHP's CuRL doesn't handle itself

+6 -4 (10 lines changed)

It wasn't actually public projects that were being listed, it just wasn't all that should have been.

There were exactly 40 pages being displayed, when there should (based on the fix) have been 72.

Gitlab returns results in a paginated format, you have to look for a next header to get the next page. The code handles that quietly in the background.

However, it seems that once in a while, Gitlab includes an extra leading space in that header value - this was being passed into PHP's CuRL which was silently refusing to connect out to an invalid URL.

I turned error_reporting and display_errors on and got absolutely nothing.

In this snippet

        curl_setopt($this->curl, CURLOPT_URL, $url);
        curl_setopt($this->curl, CURLOPT_RETURNTRANSFER, 1);

        curl_setopt($this->curl, CURLOPT_HEADERFUNCTION,
        function($curl, $header) use (&$headers)
        {
            $len = strlen($header);
            $header = explode(':', $header, 2);
            if (count($header) < 2) // ignore invalid headers
            return $len;

            $headers[strtolower(trim($header[0]))][] = trim($header[1]);

            return $len;
        }
        );


        $result = curl_exec($this->curl);

        $links = array();
        if (array_key_exists("link", $headers)){
            $links = GILSUtils::parseLinkHeader($headers['link'][0]);
        }
        return array("result" => $result, "headers" => $headers, "links" => $links);

The arrays were ending up completely empty. I dropped an echo curl_error($this->curl) in there but still nothing.

Ultimately the fix in commit c88fa2a3c was to add a trim

$url = trim($url)

to ensure any leading or trailing gumph is removed

mentioned in issue #15