I don’t know if this would fit your needs but I just have a wireguard server and when I travel I just connect to my VPN from my phone and start my access point …then all the other devices will be connected to my VPN through the phone access point (there is an option to let clients use VPN in access point settings) and watch jellyfin. For me this is the simplest and cheapest option I found.
If all the devices are in the same network you should be able to stream content even without internet. I had to wait for 5 days to my ISP set up my internet on my new address and I watched my jellyfin library in the meantime with no problems.