I know that I could probably read all documentation for Filemaker server to get answers to my question, but

Yes you can, but it will depend on security settings of that provider.

Watch out with the definition of "hosting" here. Those willing to host FM databases do so using web deployment via FMP Unlimited. I've not come across any one hosting FM Server - probably because the bandwidth requirements are too severe and the internet links are weak.

The short answer to your question

"can I connect to the db that is on the web from my local FMP and use it in the same way as an local db?"

is NO.

You would need to write your web interface first, then access the files through a web browser.

I run a networked FM Server on a 256kb WAN (Private circuit). Performance is poor at the remote site (5-6 users), but we live with.