PHPCBA-4: Server Limits may prevent a full check from being completed



Issue Information

Issue Type: Bug
 
Priority: Major
Status: Closed

Reported By:
Ben Tasker
Assigned To:
Ben Tasker
Project: PHPChangedBinaries (PHPCBA)
Resolution: Fixed (2015-01-30 12:36:35)
Components: PHP Client ,

Created: 2015-01-30 10:52:09
Time Spent Working


Description
One of my deployments has a wrapper script which regenerates additional_files.cfg on each run. The average size of this file is 4.5m (roughly 48,000 files/directories).

Runs occasionally fail due to the webhost's tweaking of php.ini .

- If safe_mode is enabled files then file_exists may return false
- The PHP memory limit may be hit
- Max execution time may also be hit

The first can only really be addressed by editing php.ini - although there are potential workarounds, they all feel a bit hacky.

The latter two can be addressed, so a bugfix should be developed.


Toggle State Changes

Activity


Have made sure each file checked can have at least 60 seconds execution time and have removed the additional_files.cfg contents from memory when no longer required.

The overall memory usage could also probably be reduced by removing files from changedbinariesmain::files as they're checked.

Running a test update/check with the current changeset at the moment.

Repo: PHPChangedBinaries
Commit: b24cca41626594eaa1b598fabe4eccf0c8a6a111
Author: Ben Tasker <github@<Domain Hidden>>

Date: Fri Jan 30 10:54:47 2015 +0000
Commit Message: Unloaded the additional_files config file from memory when no longer required. See PHPCBA-4



Modified (-)(+)
-------
changed_binaries/bin/changed_binaries.php




Webhook User-Agent

GitHub-Hookshot/afa9a7c


View Commit


Repo: PHPChangedBinaries
Commit: 307dd0d8cc0d25670d579c545fe226c618d36e22
Author: Ben Tasker <github@<Domain Hidden>>

Date: Fri Jan 30 10:59:22 2015 +0000
Commit Message: Extended time limit on a per file basis. See PHPCBA-4



Modified (-)(+)
-------
changed_binaries/bin/changed_binaries.php




Webhook User-Agent

GitHub-Hookshot/afa9a7c


View Commit


Repo: PHPChangedBinaries
Commit: 195c33e8bb6ff565215d43f3ec2e651a98b308d6
Author: Ben Tasker <github@<Domain Hidden>>

Date: Fri Jan 30 11:07:39 2015 +0000
Commit Message: Removed file/directory paths from memory once checked. See PHPCBA-4



Modified (-)(+)
-------
changed_binaries/bin/changed_binaries.php




Webhook User-Agent

GitHub-Hookshot/afa9a7c


View Commit

The test run appears to have completed successfully. Pulling in the changes in commits 1995c3e -> c5efbc8 and re-running
New test run is underway.

The changes in 1995c3e have had the expected effect - memory usage still climbs as the script progresses (it is, after all, still buffering notifications etc) but not as sharply as it did before.

The host does, however, have quite a low memory limit set for PHP (32M!) so commit f890c98 uses ini_set to up this limit to 256M (which should be more than enough!).

Repo: PHPChangedBinaries
Commit: f890c98bdbbeb9323b351c96583cca7c11b25918
Author: Ben Tasker <github@<Domain Hidden>>

Date: Fri Jan 30 11:41:59 2015 +0000
Commit Message: Set memory limit to 256M. See PHPCBA-4



Modified (-)(+)
-------
changed_binaries/bin/changed_binaries.php




Webhook User-Agent

GitHub-Hookshot/afa9a7c


View Commit

For reference, the settings that host has used are as follows

# php-cli -r "phpinfo();" | egrep 'memory_limit|max_exec|safe_'
max_execution_time => 0 => 0
memory_limit => 32M => 32M
safe_mode => On => On
safe_mode_exec_dir => /usr/local/php/bin => /usr/local/php/bin
safe_mode_gid => Off => Off
safe_mode_include_dir => no value => no value
sql.safe_mode => Off => Off
filter.default => unsafe_raw => unsafe_raw
safe_mode_allowed_env_vars => PHP_ => PHP_
safe_mode_protected_env_vars => LD_LIBRARY_PATH => LD_LIBRARY_PATH


Safe mode has since been disabled.
Commit 20fda47 prevents messages from being buffered if they won't eventually be emailed out (due to configuration settings). Messages will still be printed to stdout though.

Repo: PHPChangedBinaries
Commit: 20fda4761a5d3c7b90bf56fde20ffc55f4016d1b
Author: Ben Tasker <github@<Domain Hidden>>

Date: Fri Jan 30 12:01:25 2015 +0000
Commit Message: Prevented buffering of messages which will not be emailed. See PHPCBA-4



Modified (-)(+)
-------
changed_binaries/lib/notify.class.php




Webhook User-Agent

GitHub-Hookshot/afa9a7c


View Commit

Took some metrics before and after applying 20fda47 to the test-run.

Generated with
root@serv01 Unassigned
# for i in {1..90}; do ps aux # grep "changed_binaries.php" usage.txt USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 29086 0.8 1.0 441584 42344 pts/3 S+ 05:40 0:02 php-cli changed_binaries.php --upd root 29086 0.8 1.1 442352 42932 pts/3 S+ 05:40 0:02 php-cli changed_binaries.php --upd root 29086 0.8 1.1 442864 43488 pts/3 S+ 05:40 0:03 php-cli changed_binaries.php --upd root 29086 0.8 1.1 443888 44396 pts/3 S+ 05:40 0:03 php-cli changed_binaries.php --upd root 29086 0.8 1.1 444656 44956 pts/3 S+ 05:40 0:03 php-cli changed_binaries.php --upd root 29086 0.8 1.1 445168 45448 pts/3 S+ 05:40 0:03 php-cli changed_binaries.php --upd root 29086 0.8 1.1 445936 46044 pts/3 S+ 05:40 0:04 php-cli changed_binaries.php --upd root 29086 0.8 1.2 446192 46548 pts/3 S+ 05:40 0:04 php-cli changed_binaries.php --upd root 29086 0.7 1.2 446704 47020 pts/3 S+ 05:40 0:04 php-cli changed_binaries.php --upd root 29086 0.7 1.2 447472 47592 pts/3 S+ 05:40 0:04 php-cli changed_binaries.php --upd root 29086 0.7 1.2 447984 48216 pts/3 R+ 05:40 0:04 php-cli changed_binaries.php --upd root 29086 0.7 1.2 448496 48820 pts/3 S+ 05:40 0:05 php-cli changed_binaries.php --upd root 29086 0.7 1.2 449008 49336 pts/3 S+ 05:40 0:05 php-cli changed_binaries.php --upd root 29086 0.7 1.2 449520 49872 pts/3 S+ 05:40 0:05 php-cli changed_binaries.php --upd root 29086 0.7 1.3 450032 50456 pts/3 S+ 05:40 0:05 php-cli changed_binaries.php --upd root 29086 0.7 1.3 451828 51908 pts/3 S+ 05:40 0:06 php-cli changed_binaries.php --upd root 29086 0.7 1.3 452084 52180 pts/3 S+ 05:40 0:06 php-cli changed_binaries.php --upd root 29086 0.7 1.3 452596 52692 pts/3 S+ 05:40 0:06 php-cli changed_binaries.php --upd root 29086 0.7 1.3 453364 53232 pts/3 S+ 05:40 0:06 php-cli changed_binaries.php --upd root 29086 0.8 1.3 453620 53704 pts/3 S+ 05:40 0:07 php-cli changed_binaries.php --upd root 29086 0.8 1.4 454388 54292 pts/3 S+ 05:40 0:07 php-cli changed_binaries.php --upd root 29086 0.8 1.4 454900 54864 pts/3 S+ 05:40 0:08 php-cli changed_binaries.php --upd root 29086 0.8 1.4 455412 55376 pts/3 S+ 05:40 0:08 php-cli changed_binaries.php --upd root 29086 0.8 1.4 455668 55860 pts/3 S+ 05:40 0:08 php-cli changed_binaries.php --upd root 29086 0.8 1.4 456436 56396 pts/3 S+ 05:40 0:09 php-cli changed_binaries.php --upd root 29086 0.9 1.4 456692 56808 pts/3 S+ 05:40 0:09 php-cli changed_binaries.php --upd root 29086 0.9 1.4 457460 57452 pts/3 S+ 05:40 0:10 php-cli changed_binaries.php --upd root 29086 0.9 1.5 457972 57924 pts/3 S+ 05:40 0:10 php-cli changed_binaries.php --upd root 29086 0.9 1.4 457456 57656 pts/3 S+ 05:40 0:11 php-cli changed_binaries.php --upd root 29086 1.0 1.4 457456 57656 pts/3 S+ 05:40 0:13 php-cli changed_binaries.php --upd root 29086 1.2 1.4 457456 57668 pts/3 R+ 05:40 0:15 php-cli changed_binaries.php --upd root 29086 1.3 1.5 457456 57776 pts/3 R+ 05:40 0:17 php-cli changed_binaries.php --upd

After
root@serv01 
Unassigned
# grep "changed_binaries.php" usage.txt USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 30501 3.4 1.0 441584 42316 pts/3 S+ 06:03 0:00 php-cli changed_binaries.php --upd root 30501 2.5 1.0 441584 42316 pts/3 S+ 06:03 0:01 php-cli changed_binaries.php --upd root 30501 2.1 1.0 441584 42316 pts/3 S+ 06:03 0:01 php-cli changed_binaries.php --upd root 30501 2.0 1.0 441584 42316 pts/3 S+ 06:03 0:02 php-cli changed_binaries.php --upd root 30501 1.9 1.1 441584 42356 pts/3 S+ 06:03 0:02 php-cli changed_binaries.php --upd root 30501 1.8 1.1 441584 42356 pts/3 S+ 06:03 0:03 php-cli changed_binaries.php --upd root 30501 1.7 1.1 441584 42356 pts/3 S+ 06:03 0:03 php-cli changed_binaries.php --upd root 30501 1.7 1.1 441584 42356 pts/3 S+ 06:03 0:04 php-cli changed_binaries.php --upd root 30501 1.7 1.1 441584 42356 pts/3 S+ 06:03 0:04 php-cli changed_binaries.php --upd root 30501 1.7 1.1 441584 42356 pts/3 S+ 06:03 0:05 php-cli changed_binaries.php --upd root 30501 1.7 1.1 441584 42356 pts/3 S+ 06:03 0:05 php-cli changed_binaries.php --upd root 30501 1.7 1.1 441584 42356 pts/3 S+ 06:03 0:06 php-cli changed_binaries.php --upd root 30501 1.7 1.1 441584 42356 pts/3 S+ 06:03 0:06 php-cli changed_binaries.php --upd root 30501 1.6 1.1 441584 42356 pts/3 S+ 06:03 0:06 php-cli changed_binaries.php --upd root 30501 1.6 1.1 441584 42356 pts/3 S+ 06:03 0:07 php-cli changed_binaries.php --upd root 30501 1.6 1.1 441584 42356 pts/3 S+ 06:03 0:08 php-cli changed_binaries.php --upd root 30501 1.7 1.1 441584 42356 pts/3 S+ 06:03 0:08 php-cli changed_binaries.php --upd root 30501 1.7 1.0 440812 41848 pts/3 S+ 06:03 0:09 php-cli changed_binaries.php --upd root 30501 2.6 1.0 440812 41852 pts/3 R+ 06:03 0:14 php-cli changed_binaries.php --upd


So memory usage has been reduced considerably - the run also completed in a shorter time. For reference, that run processed 52,278 hashes.

2015-01-30 12:25:14

Repo: PHPChangedBinaries
Commit: 622e7312940aecdd85c0471c5c17c911aaefa207
Author: Ben Tasker <github@<Domain Hidden>>

Date: Fri Jan 30 12:24:17 2015 +0000
Commit Message: Adjusted notification class to ensure counts are correct even if sending disabled. See PHPCBA-4



Modified (-)(+)
-------
changed_binaries/lib/notify.class.php




Webhook User-Agent

GitHub-Hookshot/afa9a7c


View Commit


2015-01-30 12:25:21
Commit 20fda47 does, however, introduce an issue with the emailed report - the counts provided are based on counting the arrays (which are no longer being populated). Commit 622e731 implements a counter for each notification type.

2015-01-30 12:36:35
Test run completed successfully and notification count is correct. Closing this issue as Fixed.

2015-01-30 12:36:35
btasker changed status from 'Open' to 'Resolved'

2015-01-30 12:36:35
btasker added 'Fixed' to resolution

2015-01-30 12:36:39
btasker changed status from 'Resolved' to 'Closed'