When the server doesn't boot (or connect to ethernet)....
reddit / selfhosted

----------------------------------------------------------------------
immelman3 · 2023-10-05 20:25

Hello guys, I have been using a self hosted server that is hosted in a
family house where there is good connection for years now and today I
finally managed to break it. I was playing with "wake on lan" and
/etc/network/interfaces and now the ethernet doesn't autoconnect
anymore so it's not reachable via ssh...
The only solution right now is some on-site operation,am I correct? I
was wondering,what steps can I implement to avoid any future issue of
this kind?
1- Emergency bootable usb that when booted have ssh enabled by
default?
2- Any serial-over-lan solution?
3- Bios remote management?

What do you use as a workaround for such conondrum?

----------------------------------------------------------------------
clintkev251 · 2023-10-05 21:03

Probably some kind of IPKVM if the system isn't something that you
have easy access to. PiKVM is a good one to look into

----------------------------------------------------------------------
xIndirect · 2023-10-05 21:53

I have several methods to deal with this across my sites. Usually
ipkvm or IPMI of some kind. I also recently setup meshcentral / AMT
for the Intel NUC I have. Be aware AMT seems like a security nightmare
but I accept it for that host / network  and understand what it can
lead to.

There's various ipKVMs on the market such as the pikvm.  And for IPMI
solutions a lot of server motherboards have them integrated.

----------------------------------------------------------------------
mordac_the_preventer · 2023-10-05 22:19

It’s great if you have a second machine or IPKVM, but the best way
I’ve found around this is to not play with network settings when I’m
on a remote connection.

You’ll probably find that you’ve fat-fingered something in the
configuration - my favourite is spelling network as “netwokr“ or eth0
as “eht0”.

----------------------------------------------------------------------
-quakeguy- · 2023-10-06 09:40

The normal approach to this is to utilize server hardware that has an
IPMI/iDrac/similar management card that has a separate dedicated
network port with it’s own IP address. These cards allow you to cold
boot the main hardware, access the local console and basically do
almost anything to it that would normally require your physical
presense on-site (including mounting an ISO file remotely into a
virtual DVD drive and reinstalling the entire machine from scratch if
needed).

The catch is security. Since this level of access to the hardware is
basically ”god mode”, the absolutely last thing you want to do is to
have it directly visible to the public Internet. Do not do this, under
any circumstance, ever.

What you do at the bare minimum is arrange some sort of a VPN
connection to the network that this management card is connected to.
In the enterprise world, they are also placed in dedicated and highly
restricted vlans.

----------------------------------------------------------------------
lilolalu · 2023-10-06 16:52

Get a server with IPMI, SOL, KVM, vPro or whatever creative naming the
vendors come up with. Or an external KVM solution like pikvm

----------------------------------------------------------------------
immelman3 · 2023-10-06 18:54

I was looking into pikvm and it looks like having a raspberry pi 2w
connected to wifi,hdmi and usb is just what I was looking for... Super
cheap  and convenient. Thanks guys