Can you install the Red5Pro Server on a Windows VM?
You should remote desktop into the instance and run the .bat file the same way you do on the desktop.
You should be able to navigate to http://localhost:5080 locally on the Virtual Machine without opening any of the ports.
Here's what we did to install the server on the Windows VM:
1 - Logged into the machine via the Remote Desktop connection (from the Azure portal, choose the server and "connect"
2 - Opened a browser, logged into Red5Pro.com and downloaded the latest server build
3 - Unzipped the build to a folder on the desktop
4 - Double-click red5.bat (note: by default, the .bat extension is not displayed on the server ( added screen shot) - this should just open a command window and run