What is the flavor of free lunch?
When you have an urgent message for your colleague at a remote location, how do you get the message to them?
You call them, text them, email them. You try to reach their supervisor, subordinates, friends, family, etc. until you finally get through to them.
Diversification is just another word for it. You want investment returns and you are looking to achieve it in every independent way there is to achieve it. The key word here is independent. When you have an urgent message and your colleague has misplaced their cell phone, your efforts to call them and text them will be equally unsuccessful at getting the message across. Calling and texting are not independent, both depend on your colleague having access to their cell phone.
Likewise, in investing, you are looking for independent or uncorrelated return drivers and you want to invest in every uncorrelated return driver to ensure you can make a reasonable return.
Diversification by investing part of your assets in each uncorrelated return driver is the only free lunch in finance. The variability of your overall return is lowered compared to investing in just a few of these return drivers. This means when there is a shock in one part of the financial markets, the overall loss in your portfolio of assets may not be as severe as the market. The flip side is also true: when there is a strong rally in the financial markets, your portfolio of assets may not rally as strongly.
'So, if my portfolio does not swing to the fences in either direction, how can I possibly benefit?' You may ask. The way you stand to benefit with lower variability is that over the long run, a portfolio with lower variability wins over another portfolio with higher variability all other things being equal. A portfolio with lower variability compounds better over time giving you a long term advantage.