Yes you can disable the DE on Debian and probably on other distros too.

I “disable” it the wrong way, since I want to have it after a reboot, for diagnostics, I just shut it off via systemctl For gnome it should be: Systemctl stop gdm3

But you can also configure it to always start headless. And if you need it you can launch it again from the shell.