Page load speeds are an important factor in both search engine rankings and human userbility. A safe page load speed to aim for is under 2 seconds (you can check the current speed of your site here). Fortunately Joomla has a number of options to help you speed up your site that are turned off by default.
Cache your site !!
It works by saving a copy of the page in Joomla's cache folder each time it is loaded every X amount of minutes and it is the copy that is loaded on the next visit rather than having to generate the whole page again. The amount of minutes can be specified by yourself and is entirely down to how often you feel the site is updated and want the cache to refresh.
The drawback of this is that if you are editing your site you will not see any immediate changes take effect until the cache has refreshed (or until you manually clean the cache). Additionally for more dynamic sites some features (such as the random image module or a comments facility) will not appear to work due to the cache "freezing" the previous appearance. In most cases however these modules provide a "no cache" parameter (under the advanced parameters section) that overrides the global settings to avoid freezing these features.
The cache option can be broken down into three specific sections (Global Cache, System Cache & Browser Cache) each of which can be activated independantly.
The global cache stores copies of the template view, the article view and individual module views (each of which are treated seperately). To enable the global cache go to Site > Global Configuration > System. The cache settings are on the right, click on "Yes" to enable it and then specify your cache lifetime (in minutes) in the box below.
In contrast to the global cache the system cache stores copies of whole page views. To enable the system cache go to Extensions > Plugin Manager then click on "System Cache" and tick the enable button. You can specify the cache expiry time (in minutes) and whether to enable browser caching or not.
Browser caching can be enabled on the same page as the "System Cache" plugin. It simply informs the clients web-browser that the content has not been updated and therefore can use the files stored locally in the browser's cache rather than downloading them all over again. This only affects pages that have already been visited and is entirely dependent on the visitor's browser settings.
Clean the Cache
If you ever need to clean the cache simply go to Tools > Clean Cache. This lists all the cached elements (including module views if you have enabled the global cache) and each can be selected and deleted using the trash can at the top of the screen.
Gzipping a site compresses the HTML content into the smallest size possible, the browser then decompresses it (when viewed) at a speed that can be considerably quicker than downloading the original uncompressed HTML. However this only works if your web-host supports Gzip and is only beneficial if your site is quite HTML heavy. Compressing files can cause a slightly higher load on the server, so if your site isn't HTML heavy then you risk increasing the server load for a speed difference that is barely noticeable. Compression is normally about 4:1 but again depends entirely on how much HTML you have on the page (images are not included in the Gzip process). You can check if your webhost supports this and what difference your website speed will be here.
To enable Gzip go to Site > Global Configuration > Server then click on the enable button in the "Gzip site compression" section to the left.