Hello! I (tried, at least) converted an old laptop to a Debian home server, and I was trying to set up duckdns.org and to enable port forwarding on my router. internet connection was working, I installed packages, docker, immich, etc, and then suddenly (I don’t know exactly when) it refuses to connect to the internet. It does connect to local addresses (I can ssh into it) but ping google.com
and any other internet-involving command fail. I had set up a rule on the router to forward port 80 to the device’s port 80, but I then removed the rule and it still does not connect to the internet. I rebooted the router but nothing changed. Any idea what could be? the router is a Vodafone router.
I changed the hostname to debianserver but on the router it is still written debian. Also, it’s the only device with unknown ipv6
thanks in advance!
EDIT: I rebooted again the server, and now ipv6 is not unknown anymore, and the hostname is correct. however, it still does not connect to the internet
EDIT 2:
only one device (debianserver) has this problem, other devices work as before
EDIT 3:
I don’t know if it’s useful or not, but if I boot a live debian USB in the server internet works
SOLUTION: aaaaand no it does not work, after restarting docker it seems to work because all the brodges are yet to be created and it takes some time, after like 30 seconds it does not work as before :(
- edit the file
/lib/systemd/system/docker.service
- append the flag
--bip=192.168.3.1/24
toExecStart=....
systemctl daemon-reload
systemctl restart docker
docker was the fucker that messed everything up and made me lose a few hours!
EDIT 4:
it seems that ip route flush 0/0
restores the internet connectivity until reboot… I don’t know what does this means but can be a temporary workaround I guess? I really have no idea how to solve this
FINAL EDIT: I gave up. I removed debian and installed fedora, and now it all works like a charm
Does
ping 8.8.8.8
work? (To check if it’s DNS)What’s
ip route show
say? (Just to try to narrow down whether it’s an issue with the server’s config or the router’s)What’s
traceroute 8.8.8.8
display?thanks for the answer!
ping 8.8.8.8
fails, and I don’t have traceroute installed (and no internet to install it)tubbadu@debianserver:~$ ip route show 0.0.0.0 dev veth3492bf7 scope link 0.0.0.0 dev vethc1bf668 scope link 0.0.0.0 dev vethb41fd7e scope link 0.0.0.0 dev veth2e39932 scope link 0.0.0.0 dev veth68451d9 scope link default dev veth3492bf7 scope link default dev vethc1bf668 scope link default dev vethb41fd7e scope link default via 192.168.1.1 dev enp1s0 169.254.0.0/16 dev veth68451d9 proto kernel scope link src 169.254.210.75 169.254.0.0/16 dev veth2e39932 proto kernel scope link src 169.254.242.12 169.254.0.0/16 dev vethb41fd7e proto kernel scope link src 169.254.185.90 169.254.0.0/16 dev vethc1bf668 proto kernel scope link src 169.254.225.22 169.254.0.0/16 dev veth3492bf7 proto kernel scope link src 169.254.123.220 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 172.18.0.0/16 dev br-56cf32fc7cde proto kernel scope link src 172.18.0.1 192.168.1.0/24 dev enp1s0 proto kernel scope link src 192.168.1.9 192.168.1.1 dev enp1s0 scope link
This is kind of a nutty network config. It looks like docker is setting up extra default routes, which I could easily see fouling everything up. As a first experiment (warning, this may ruin your networking until the next reboot):
ip route flush 0/0 ip route add default via 192.168.1.1 dev enp1s0
… and see if that makes things work (start with
ping 192.168.1.1
andping 8.8.8.8
). If that solves the problem, then I think something about your docker config is adding stuff to your networking that’s causing the problem; maybe remove/disable docker completely and then re-add docker things one at a time to see where the problem comes in.okay, I thought to have solved the problem but I was wrong, here I go again. When I
docker compose up -d
the immich server (the only one I have installed) all those routes are created, and apparently some of them conflicts with something else and now my host has no internet connection. however it seems thatip route flush 0/0
solves the problem until the reboot, which is strange. the other command returnsRTNETLINK answers: File exists
Hm. Yeah, that’s weird. The default routes you’re seeing should basically never exist, so it sounds like there’s some kind of manual network config happening inside the Docker container that’s creating a broken network.
What does
docker network inspect [network]
say for each of your Docker networks (substituting each Docker network for “[network]”)? What’s the network section ofdocker-compose.yml
look like?
Try removing all the superfluous default routes.
My guess is that you have Docker configured incorrectly. Its internal IP range probably overlaps with your real network, so all requests are routed to Docker. Uninstall docker and reboot the server. If that works, reinstall docker and properly configure its internal networking.
Go to the outer status page. The router should display whether it has an internet connection to your provider. If no, then your router/modem has no credentials or another issue preventing access.
If it shows as working, then you can narrow it down to incorrect DNS and IP routing. Perhaps dynamic IP allocation is set to off or another configuration error or bug, in which case you might need to reset all the router settings. Then, is it only broken for a single end device?
thanks for the reply! Sorry it isn’t very clear from the post, but yes only one device (debianserver) has this problem (no internet connection, but yes local network connection), all other devices works as before. I’ll update the original post to clarify this
I understand you might be a bit stressed out, but it’s very hard to make head or tails of what you’re describing.
E.g. What do you mean by “no internet connection”? Does it mean just the old laptop, all devices on the network or the router itself?
thanks for the answer! Sorry I didn’t specified very well: only the server has problems, all other devices work as before
I bet it has something to do with the LAN DHCP setup, since you have to set a static IP for the server…
Update - That won’t work at all if it is indeed the issue. Let me think for a sec on how to pull external ip by doing a IP wget…
wget -qO - icanhazip.com
it waits like 10 seconds and then exits without any output
Yeah, I realized after the fact. This should work:
wget 104.18.114.97
, and you should get a ERROR 403: Forbidden if your server has any internet connection.tubbadu@debianserver:~$ wget 104.18.114.97 --2024-01-18 21:35:38-- http://104.18.114.97/ Connecting to 104.18.114.97:80... failed: No route to host.
really really thanks for the help!
Please try
wget https://104.18.114.97
, if this one goes through I’d think the problem could be related to a faulty forwarding of regular internet traffic (port 80). If that fails as well I’d guess it’s to due with the DHCP/Static IP’s and involves your router. I’m absolutely clueless about Vodafone routers though.Edit: Any connection would show “The certificate’s owner does not match hostname ‘104.18.114.97’”
same as before:
tubbadu@debianserver:~$ wget https://104.18.114.97 --2024-01-18 21:52:29-- https://104.18.114.97/ Connecting to 104.18.114.97:443... failed: No route to host.
Edit: Any connection would show “The certificate’s owner does not match hostname ‘104.18.114.97’”
sorry I don’t think I understand, where should this be written?
You mentioned you changed firewall rules for that device. Any chance you have set outbound rule instead of inbound rule?
Anyway, what’s the output of
ip route
?Anyway, what’s the output of ip route?
tubbadu@debianserver:~$ ip route 0.0.0.0 dev veth60f206e scope link 0.0.0.0 dev veth48a6716 scope link 0.0.0.0 dev vethdf7381d scope link 0.0.0.0 dev veth59e46aa scope link 0.0.0.0 dev vethac4830e scope link default dev veth60f206e scope link default dev veth48a6716 scope link default dev vethdf7381d scope link default dev veth59e46aa scope link default via 192.168.1.1 dev enp1s0 169.254.0.0/16 dev vethac4830e proto kernel scope link src 169.254.241.201 169.254.0.0/16 dev veth59e46aa proto kernel scope link src 169.254.97.105 169.254.0.0/16 dev vethdf7381d proto kernel scope link src 169.254.212.49 169.254.0.0/16 dev veth48a6716 proto kernel scope link src 169.254.26.74 169.254.0.0/16 dev veth60f206e proto kernel scope link src 169.254.212.242 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 172.18.0.0/16 dev br-56cf32fc7cde proto kernel scope link src 172.18.0.1 192.168.1.0/24 dev enp1s0 proto kernel scope link src 192.168.1.9 192.168.1.1 dev enp1s0 scope link
You mentioned you changed firewall rules for that device. Any chance you have set outbound rule instead of inbound rule?
All I did was doing this and press Save:
(now I rewrote it just to take the screenshot)
after internet stopped working I edited it to set the port to 443, and then I removed the rule. there are no rules now
Is there a reset button somewhere on the router? Most of them have something like this in order to reset them to factory settings. If not, google for your device name and factory reset, maybe it’s something like „press button while turning on“ etc. I’d try something like that
Yes there is in the web UI a factory reset button, but I’d rather not do this because it has some settings by other people
So, several people share the router and can modify settings…
That’s a recipe for trouble.I do external support for small businesses. When I work, configure etc. sometning, I change the password. I write the password on a piece of paper and put it in an envelope. Then I sign across the sealed flap. That envelope is handed to the customer. If I cease to work for them or something happens to me, they can open it and retrive the password. If something goes belly up, I ask for the envelope. If the envelope has been tampered, I wash my hands, and chrge handsomly to solve the issue.
I’d reset it, and then make someone responsible for doing things to the router, that way everyone knows what’s going on.
whel this is clever XD