########################################################################################## MISC-24: Samsung Galaxy S4 slow to start establishing connections ########################################################################################## Issue Type: Bug ----------------------------------------------------------------------------------------- Issue Information ==================== Priority: Major Status: Closed Resolution: Done (2019-07-16 16:04:23) Project: Miscellaneous (MISC) Reported By: btasker Assigned To: btasker Environment:Android 5.0.1 Samsung Galaxy S4 GT-I9505 Affected Versions: - Samsung Galaxy S4 Issues Targeted for fix in version: - Samsung Galaxy S4 Issues Labels: android, connectivity, wifi, Time Estimate: 300 minutes Time Logged: 60 minutes ----------------------------------------------------------------------------------------- Issue Description ================== I had to replace my original Samsung Galaxy S4 fairly recently (having accidentally generated a small field and frying a few components... long story). The new phone is very slow to connect when on wifi, and has been since I got it (only just got around to starting to look into it properly though). Having run packet captures, it's not the connection itself that's slow, but the phone seems to wait around 5 seconds before the SYN even hits the wire. After that the connection is fine. Not seeing any signs of packet loss. Phone isn't rooted (previous one was), and I've previously had both "No Root Firewall" and "OpenVPN Connect" installed on it. The issue seems to affect all connections rather than just a specific browser or something like that. Seems to consistently be 5 seconds, suggesting it's waiting for something to time out before it actually starts trying to establish the connection. Where a site calls resources from multiple domains/servers (by, for example, using a CDN), that can easily translate into a 30 second wait for the page to load fully. Raising this issue to track troubleshooting, as there don't seem to be any resources on Google to help narrow this issue down - most are concerned with the more commonly known issues with wifi on the S4. I've ruled those out: - Keep Wi-Fi on during sleep is enabled - WIFI Power Save Mode is off Another S4 on the network (running a slightly older version of Android) doesn't have this issue. Doesn't mean it definitely isn't something on the network, but as far as I can see, the network is running fine. The phone does seem to behave on 4G though. ----------------------------------------------------------------------------------------- Issue Relations ================ - relates to LAN-127: Revisit LAN IPv6 support - No Root Firewall (Google Play) (https://play.google.com/store/apps/details?id=app.greyshirts.firewall&hl=en_GB) - OpenVPN Connect (google play) (https://play.google.com/store/apps/details?id=net.openvpn.openvpn&hl=en_GB) ----------------------------------------------------------------------------------------- Activity ========== ----------------------------------------------------------------------------------------- 2018-01-06 11:49:28 btasker ----------------------------------------------------------------------------------------- One thing I wondered about (but have since eliminated) was whether the phone was SLAACing an IPv6 address. I do make IPv6 available to a couple of devices on the network, but limit IPv6 egress to those machines. So wondered if it was trying to establish an IPv6 connection first, waiting for that to timeout (as it won't get past the firewall) and then falling back to IPv4. The phone's settings don't show a global IPv6 address though (there is a link local), and I'm not seeing an IPv6 traffic from it in captures, so I'm reasonably sure it's not that. I did also wonder whether No Root Firewall or OpenVPN Connect had perhaps left something lying around that needed to timeout, but both have been uninstalled and the phone rebooted, so I don't think it's that (and if it is, it's going to be an absolute bastard to find and identify) The system cache has also been wiped since then, so it's unlikely that anything is lying around. ----------------------------------------------------------------------------------------- 2018-01-06 12:01:24 btasker ----------------------------------------------------------------------------------------- It looks like there may be something odd going on with DNS though. In a terminal emulator, if I run a ping to Google's DNS - ping -c 3 8.8.8.8 - the ping starts running immediately. However, if I ping a hostname - ping -c3 astria.sys.bentasker.co.uk - it takes a few seconds for it to start. The only thing is, if I run test resolutions from within the DNS Lookup app, it's instant, whether using the same server as configured via DHCP, or Google's etc. I'll run some captures of DNS traffic to see if anything stands out (because [Its Always DNS|https://imgur.com/YKvOiA4]). I do have various egress rules to limit DNS leakage, so it may relate to that. ----------------------------------------------------------------------------------------- 2018-01-06 12:10:54 btasker ----------------------------------------------------------------------------------------- Oh, FFS In the capture, I can see a DNS lookup for the domain I tested against, going to my LAN's DNS server. Android is using the local scope IPv6 address in order to try and perform lookups. The DNS server (of course) doesn't know where the hell to send the response, so doesn't reply. Once the lookup times out on the phone, it tries again over IPv4. That's stupid. I don't actually provide the DNS server's IPv6 address in DHCP responses, so presumably the phone is managing - in part - to SLAAC. At least in so much as it receives the DNS server's IPv6 address back. Options to fix are - Disable IPv6 on the phone (requires rooting) - Block it from SLAACing at all (thought it already was though, so this may not be possible) - Set up the config so it can SLAAC and use IPv6 - DNAT and SNAT the packets on the DNS server to force it back to IPv4 The easiest is probably going to be option 3, option 4 is a horrible hack (plus, even if it wasn't, I don't think iptables currently supports NAT'ing IPv6 to IPv4). ----------------------------------------------------------------------------------------- 2018-01-06 12:37:53 btasker ----------------------------------------------------------------------------------------- Still not working. Helpfully, this thread - https://forum.pfsense.org/index.php?topic=116055.0 - would suggest that Samsung may have screwed the pooch with their IPv6 support. So I might instead need to look at more effectively blocking it from SLAACing. The OP in that thread is at least reporting the same issue as me - slow loads because of having to wait for DNS to timeout. Means this might be a real pain to fix. Just to prove the issue, I've turned off Router Advertisement (RA) on the router and re-run the test. Connectivity is instant (as it should be) and packet captures show DNS lookups from the phone are IPv4 from the outset. Now just need to find a proper fix :( ----------------------------------------------------------------------------------------- 2018-01-06 12:50:40 btasker ----------------------------------------------------------------------------------------- OK, I've set the RA mode to Assisted (from Managed) and the phone now seems to actually get a global scope address, so everything works. Need to run a new capture to double-check that it's not an issue with the router when in managed mode (though everything else works, so it's probably a reasonable assumption that it's just yet another Samsung screw-up). ----------------------------------------------------------------------------------------- 2018-01-06 13:07:17 btasker ----------------------------------------------------------------------------------------- Everything looks OK in the captures, so I guess it is a Samsung issue. CCtrl 177 refers to the change on the router ----------------------------------------------------------------------------------------- 2018-01-06 13:07:21 ----------------------------------------------------------------------------------------- btasker changed status from 'Open' to 'Resolved' ----------------------------------------------------------------------------------------- 2018-01-06 13:07:21 ----------------------------------------------------------------------------------------- btasker added 'Fixed' to resolution ----------------------------------------------------------------------------------------- 2018-01-06 13:07:24 ----------------------------------------------------------------------------------------- btasker changed status from 'Resolved' to 'Closed' ----------------------------------------------------------------------------------------- 2018-01-06 13:07:54 ----------------------------------------------------------------------------------------- btasker removed 'Fixed' from resolution ----------------------------------------------------------------------------------------- 2018-01-06 13:07:54 ----------------------------------------------------------------------------------------- btasker changed status from 'Closed' to 'Reopened' ----------------------------------------------------------------------------------------- 2018-01-06 13:08:13 ----------------------------------------------------------------------------------------- btasker changed timespent from '0 minutes' to '60 minutes' ----------------------------------------------------------------------------------------- 2018-01-06 13:08:18 ----------------------------------------------------------------------------------------- btasker changed status from 'Reopened' to 'Resolved' ----------------------------------------------------------------------------------------- 2018-01-06 13:08:18 ----------------------------------------------------------------------------------------- btasker added 'Fixed' to resolution ----------------------------------------------------------------------------------------- 2018-01-06 13:08:22 ----------------------------------------------------------------------------------------- btasker changed status from 'Resolved' to 'Closed' ----------------------------------------------------------------------------------------- 2018-02-15 11:31:38 btasker ----------------------------------------------------------------------------------------- It still occasionally decides it's going to use IPv6 (looks like it tends to be after it's been on 3/4G and got IPv6 config there). So I've now added a rule to the firewall to reject all IPv6 packets that originate from the link-local reserved range ( fe80::/10 ) and thing seems to be loading in a semi-reasonable amount of time again. Still slower than I'd like, but a marked improvement none-the-less. I really don't want my phone (and particularly other devices) using IPv6 at home, as there are various things (split-tunneling, filtering etc) that I haven't got around to configuring for V6 yet. There also isn't currently really any personal advantage to using IPv6 at the moment either, I'm *never* going to allow direct connections to any of the devices on the LAN and there isn't really anything IPv6 only that's overly compelling at the moment. The only advantage I can see at the moment is it would fix this stupid behaviour, which patently shouldn't exist in the first place. If Android at least supported DHCPv6 I could prevent this behaviour fairly easily, but as it doesn't options are fairly limited. Firewalling the link-local addresses only masks the symptoms, but as far as I can see the only real _fix_ would be to set up a new access point in a VLAN with no access to any of the IPv6 infra. Alternatively, I could look at what of my IPv6 using kit actually *needs* IPv6. I don't think much of it does, as I largely set it up, just because. So one possibility might also be to tear down the IPv6 infra and go back to IPv4 only. Also not really ideal. There's *got* to be a better fix than the above options ----------------------------------------------------------------------------------------- 2018-02-15 11:31:44 ----------------------------------------------------------------------------------------- btasker removed 'Fixed' from resolution ----------------------------------------------------------------------------------------- 2018-02-15 11:31:44 ----------------------------------------------------------------------------------------- btasker changed status from 'Closed' to 'Reopened' ----------------------------------------------------------------------------------------- 2018-02-15 11:55:57 btasker ----------------------------------------------------------------------------------------- Also, looking at the address being used, I've just noticed that IPv6 Privacy Extensions aren't even being used :( Address randomization came in in Android 4 (and I'm running significantly newer than that), but I _guess_ it probably only fires against global scope addresses. It does, however, mean that there's a (more or less) fixed specific address I can block rather than killing everything in the link-local range (though it'd be handy to log for a while and see if there's anything else doing it, not noticed anything so far) ----------------------------------------------------------------------------------------- 2019-07-16 16:04:23 ----------------------------------------------------------------------------------------- btasker changed status from 'Reopened' to 'Resolved' ----------------------------------------------------------------------------------------- 2019-07-16 16:04:23 ----------------------------------------------------------------------------------------- btasker added 'Done' to resolution ----------------------------------------------------------------------------------------- 2019-07-16 16:04:27 ----------------------------------------------------------------------------------------- btasker changed status from 'Resolved' to 'Closed' ----------------------------------------------------------------------------------------- Worklog ======== ----------------------------------------------------------------------------------------- 2018-01-06 13:08:13 btasker 60 minutes ----------------------------------------------------------------------------------------- Troubleshooting, smoking, swearing and fixing