Website slow to load after some seconds of inactivity

Indigo

@thomascharbit thanks for adding your input. I was rather hoping this was a one-off but with more than one person seeing this right now hopefully we can get to the bottom of it more easily. Same as for @countfak if you have the time to try a clean WP install that would be great. If clean is fine, I guess try adding plugins/themes until you see the issue?

Would either of you say this is new behaviour in Indigo?

countfak

Indigo Would either of you say this is new behaviour in Indigo?

I noticed this behaviour when I started using it regularly for development as my main local server environment, which was just a couple of weeks ago, so I wouldn't know if this used to happen before.
Before then I didn't use it long enough to notice.

thomascharbit

Hi, i just tried with a fresh new WordPress install and i get the same result: fast page loading if reloaded every few seconds, but get down to 10-15 secs if i wait for like 30 seconds before reloading the page.

Fast (150ms)

Slow (12s)

thomascharbit

This is actually not related to WordPress or even PHP, as i get the same results if server a simple index.html file from the document root.

Indigo

Woah, mind blown lol Nice find @thomascharbit

Indigo

@thomascharbit can you confirm that this happens if your stack literally has nothing other than one nginx in it, pointing at an index.html file on your Mac's internal SSD?

Can I ask what your test site's domain name is? Or even better if you could zip the test stack and send it to me at hello@indigostack.app

Indigo

Actually, a few simple tests could be interesting also:

if you're not already, try using a simple domain name such as mysite.test
try accessing the site by its assigned port eg https://mysite.test:50001 (bypasses the nginx reverse proxy)
try accessing the site by localhost eg https://localhost:50001 (bypasses domain name resolution)

That said, my simple apache + index.html test stack usually returns a response in 1–3ms but I saw 70ms once, then immediate reloads were 1–3ms again, so I wonder if I have actually just seen a similar effect as you're describing, but just so much faster that it's not humanly noticeable? I'll check mine again and see how replicatable it is.

thomascharbit

My test domain is https://plainhtml.test.
I just tried https://plainhtml.test:50001 but this is not working (getting This site can't be reached)
https://localhost:50001/ not working aswell..
Sorry i misread, wrong port used here.

thomascharbit

So still getting the same issue when bypassing the reverse proxy.
But issue fixed if bypassing domain name resolution.

I have been using other tools like Valet+ previously, so dnsmasq was in use, not sure if this might be related.

Indigo

@thomascharbit thanks for your help getting to the bottom of this. It's very useful narrowing it down to DNS resolution.

I believe Valet registers *.test so if you have a chance I'm interested to see how you go with a non-.test domain. Perhaps plainhtml.example instead.

Also, could you see if dnsmasq is currently running on your Mac? (check the Activity Monitor—make sure you have "All processes" selected in the "View" menu).

thomascharbit

Switching to plainhtml.example works.
Dnsmasq is not running, i am unsure what Valet does with *.test domains that would cause this.

Indigo

That's excellent progress. @countfak — do you use a .test domain name? If you do, I have a couple of questions for you:

could you try changing the domain name as above? I am hoping you'll see the same improvement with a non-test domain.
have you ever installed Valet or another such system?

thomascharbit i am unsure what Valet does with *.test domains that would cause this.

The Mac's DNS resolution mechanism is complex indeed. If you don't mind me hypothesising: I suspect Valet registered Dnsmasq as a responder for the *.test domains, so when presented with a .test domain your Mac's DNS resolution system is waiting to see if Dnsmasq responds, which it won't because it's not running. After it times out your Mac then moves onto other responders including the one which reads /etc/hosts where Indigo's domains are registered; it then likely caches this DNS result for a fixed period of time, within which it will return a very quick DNS resolution.

I'll dig into how Valet registers Dnsmasq and see if there's something that can be cleaned up to remove its influence.

countfak

Indigo do you use a .test domain name?

Hi, I always use a .local domain name because that's what I'm used to. Anyway, I just tried with a .example domain name with a new stack that only has nginx with a simple html file. In this case everything works fine.
Then I also tried with one of the stacks that give me problems, changing the domain name to .example, and even in this case it seems like the issue is gone.

Weirdly enough, the last time I used Indigo for some time with one of my active projects, everything worked fine even with the .local domain name: no slow loading time, even when waiting more than like 30 seconds. Today, if I give this a try with a .local domain name, I get slow loading times. So I really have no clue... Does switching domain names do the trick? Is the issue random? Am I missing something?

Indigo have you ever installed Valet or another such system?

No I never used Valet. I used to have Local by Flywheel which used .local domain names, but now I don't have it anymore, and I shouldn't have leftovers from it on my mac.

As I said in another discussion, I'm using NextDNS.

Thanks @thomascharbit for testing.

thomascharbit

I somehow managed to fix it with *.test domains; not sure how.
I uninstalled Valet, tried a few things in my mac DNS settings.

Before i had the following DNS lookup:

; <<>> DiG 9.10.6 <<>> plainhtml.test
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 63288
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;plainhtml.test.	IN	A

;; AUTHORITY SECTION:
.			3241	IN	SOA	a.root-servers.net. nstld.verisign-grs.com. 2023052600 1800 900 604800 86400

;; Query time: 29 msec
;; SERVER: 2a01:cb04:87f:4300:8ec5:b4ff:fe90:6490#53(2a01:cb04:87f:4300:8ec5:b4ff:fe90:6490)
;; WHEN: Fri May 26 13:45:49 CEST 2023
;; MSG SIZE  rcvd: 125

Now i get:

; <<>> DiG 9.10.6 <<>> plainhtml.test
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 41552
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;plainhtml.test.		IN	A

;; Query time: 7 msec
;; SERVER: 2a01:cb04:87f:4300:8ec5:b4ff:fe90:6490#53(2a01:cb04:87f:4300:8ec5:b4ff:fe90:6490)
;; WHEN: Fri May 26 13:45:55 CEST 2023
;; MSG SIZE  rcvd: 50

Indigo

Interesting that in both cases, another system (Local by Flywheel and Valet) has installed Dnsmasq to resolve a catchall TLD, and that in both cases only domains in that catchall TLD are causing the error:

@countfak had Local by F, which used Dnsmasq to resolve *.local; configuring something.local in Indigo exhibits the problem
@thomascharbit had Valet, which used Dnsmasq to resolve *.test; configuring something.test in Indigo exhibits the problem

Described that way, the problem certainly seems outside of Indigo, but of course finger-pointing at Local/Valet doesn't help anyone 🙂 I'll do a bit more research — my current thinking is there must be a legacy configuration that both those systems have left on your respective Macs.

thomascharbit I uninstalled Valet

Had you previously uninstalled Valet? If not, I suppose it lends weight to my theory above. Anyway, great news that you resolved it!

Natrim

for me helped to remove the leftover dnsmasq *.test resolver and restarting mac dns
sudo rm /etc/resolver/test
sudo dscacheutil -flushcache && sudo killall -HUP mDNSResponder

Indigo

Thanks @Natrim, that's very helpful. I'm a bit disturbed that so many people have had this issue (or similar). However I guess it's outside of scope for Indigo to really do anything about it.

Can I ask what you used that configured dnsmasq to resolve *.test? Was it Valet?

Natrim

yep, it was probably from Valet, as i used it before

probably would be nice if Indigo checked /etc/resolver folder for used domains and warned about it when using same domain in site as it seems many ppl had this problem and many install scripts tell you to add it to /etc/resolver (ie. https://gist.github.com/pandaskii/13744293770e18d454de46cad88127ce and https://gist.github.com/petemcw/9265821)

Indigo

Hmm, that's an excellent idea and should be easy enough to implement. I've put it on my list, thanks!

countfak

Following up in case this is helpful.
My /etc/resolver directory was empty, no sign of the .local domain name there.
Instead, I ran the sudo dscacheutil -flushcache && sudo killall -HUP mDNSResponder and looks like everything is working normally until then: no slow loading times.
I'll keep you posted.