The 5-minute (or less) WordPress installation is definitely not a problem for you. Some hosting provider even provide you with a one-click WordPress installation. So installing is no always the question about WordPress, the question is to make sure you can maintain it for a long run.
There are lots of ways to optimize WordPress, but here we list a very simple and quick tips you can use.
Use limited plugins

Let’s install a sharing plugin. Hmm… Now let’s go with the Facebook like box plugin, well, how about the Twitter widget plugin? Does this sound like you? Well, don’t do that. A lot of “widget” can be done with the respective website itself.
For Facebook, you can use their online tool to generate the widget; you can generate Twitter widget too at your profile page.
Install a cache plugin

A web cache plugin is a life saver for almost all WordPress website. A good plugin such as WP Super Cache should be enough to save you some precious bandwidth.
Monitor your plugins

Some plugin may have act up, it is best to check your plugin moderately or you can use P3 plugin to help you.
Optimize your images

Don’t worry about how to optimize your image in Adobe Photoshop, don’t even worry about you don’t have Adobe Photoshop. There are lots of tools online and application that can help you quickly with your image.
First you can use this online tool to adjust your image, it’s like an online version of Photoshop. After you edit your image, you have this little online tool that compress and optimize your image.
Choose your theme carefully

Seriously, some themes doesn’t really care about WordPress standard. The reason there is a standard set onto theme is because there are issues when the standard is not followed. Always look for a theme that compile with WordPress standard.
In conclusion, a website need to be maintain, be it a WordPress powered or not. Each and everything web related is very important, cache, code standard and etc. It is always good to maintain your website from time to time, be it just a clean up on your hosting account or removing unnecessary files.