Yes, you own the website after full payment is received.
You own the copyright to the site except for any proprietary or Open Source software. Likewise, with stock images; for any we supply, you will be licensed to use them, but you won’t own the copyright.
Should you wish to move to another web host in the future, you are free to take everything – we can package your complete site and database as a zip file for your new host to extract and install on their server. However, these days most will install a plugin for this task so that they can extract the site and database in one go. Alternatively, they may use SSH, or if they are very old-fashioned, they can use FTP. All options are available, and you are not locked in.