Algae can appear as green, black, or brown patches on the fence. Musty or damp smell: If the fence is affected by algae or mould, you might notice a musty or damp smell.