My experiences are few years old, so I don’t remember excact models anymore, but some back-ups models (es series rings a bell, but as I said, it’s been a while) had batteries with soldered connectors and form-factor which (at least at the time) wasn’t available from anyone else than APC.
Bare metal server sounds like optimal solution for you and set up a hypervisor on top of it, so it’s pretty trivial to migrate VMs to your own hardware when needed. But then for your ‘long term’ environment VPS would most likely be better and migrating a full VM from your hypervisor to VPS is a bit more work, but can be done.
I don’t know about providers in Australia, but Hetzner has both and combined billing and my personal experience with them is pretty good. But I’m in Europe, so bandwidth nor latency is not a problem.