We attempt to interpret existing data on the evolution of the UV luminosity function and UV colours, $\beta$, of galaxies at $5 \leq z \leq 8$, to improve our understanding of their dust content and ISM properties. To this aim, we post-process the results of a cosmological hydrodynamical simulation with a chemical evolution model, which includes dust formation by supernovae and intermediate mass stars, dust destruction in supernova shocks, and grain growth by accretion of gas-phase elements in dense gas. We find that observations require a steep, Small Magellanic Cloud-like extinction curve and a clumpy dust distribution, where stellar populations younger than 15 Myr are still embedded in their dusty natal clouds. Investigating the scatter in the colour distribution and stellar mass, we find that the observed trends can be explained by the presence of two populations: younger, less massive galaxies where dust enrichment is mainly due to stellar sources, and massive, more chemically evolved ones, where efficient grain growth provides the dominant contribution to the total dust mass. Computing the IR-excess - UV color relation we find that all but the dustiest model galaxies follow a relation shallower than the Meurer et al. (1999) one, usually adopted to correct the observed UV luminosities of high-$z$ galaxies for the effects of dust extinction. As a result, their total star formation rates might have been over-estimated. Our study illustrates the importance to incorporate a proper treatment of dust in simulations of high-$z$ galaxies, and that massive, dusty, UV-faint galaxies might have already appeared at $z \lesssim 7$.