All ports of a VM instance filtered/blocked after stopping/starting

Google Cloud Compute Infrastructure
1

I am working with a VM running Ubuntu. After stopping and starting the VM it turned out, that ssh access to this instance from the internet is not possible anymore. Further investigation using nmap launched from a VPS located in the internet revealed that all ports of the instance are filtered/blocked, although all firewall rules defined for this instance seem to be o.k. and worked beforehand.

The status of the instance according to the Cloud console is “running”.

What is the procedure to find the root cause for this issue and to rectify it.

1 Like
2

Hello @wobits ,

are you able to log in to this VM via serial port ? It looks like ssh config is corrupted. You can also try to mount this boot volume to another Vm and check ssh config. Mounting boot disk to second VM is my way of fix if I must fix something at VMs. Btw, are you getting any error when attempting ssh connection ?

cheers,
DamianS

1 Like
3

Hello Damian,

I can launch the serial console, but the I am asked for credentials. I assume that the login is root, but I do not know the password. I do not know how to set one.

You can also try to mount this boot volume to another Vm and check ssh config.

I have no idea how to do this. I need more details on this.

1 Like
4

Hi,

  1. Stop broken VM.
  2. Detach boot disk ( would be good to remember name of this boot disk)
  3. Create or use VM instance created in the same zone as broken boot disk has been created)
  4. Attach this broken bootdisk as external disk
  5. Boot VM.
  6. fdisk -l should show you your disks. Choose this with type Linux filesystem

DamianS_0-1709704910704.png
DamianS_0-1709704910704.png612×270 20.9 KB

  1. Create mountpoint mkdir /data
  2. Mount brokenboot disk as normal partition mount /dev/sda1 /data

DamianS_1-1709704950533.png
DamianS_1-1709704950533.png774×321 10.4 KB

  1. Once mounted, you can deal with ssd config.
  2. Your data from broken bootdisk would be mounted at /data and you should modify data from that location.
  3. Additionally, please attach logs/error which you are getting during login.

Additionally, you can edit:

  • /data/etc/passwd
  • /data/etcs/shadow
  • /data/etc/groups
    To add user locally without using useradd command, and then after reattaching broken bootdisk to you primary VM, login via serial console as this user. Then editing sshd will be easier.
    How to add local user in this way? You should know how /etc/passwd and /etc/shadow files structure looks (tons of info under the Internet).

Additionally:
You can create .ssh directory for your user at the broken bootdisk and configure ssh to accept ssh key taken from your laptop. There is a tons of tutorials, how to configure such.

cheers,
DamianS

1 Like