I've been using 5 gallon buckets for water storage for some time. Haul 'em up the mountain for camping, stack 'em under the garage for emergencies. A few drops of bleach and they keep for weeks. Plus, they nest when empty, which is nice.
I used to scrounge containers. Empty kitty litter jugs, old oil jugs from restaurants, that sort of thing. Those all photo degrade pretty quickly, though. Had 3/4 of my water supply rupture on a trip to the campsite once. After that, I invested in 5 gallon buckets.
Do note that while bleach kills almost everything, it doesn't bother green algae.