Boiling water for drinking is probably not convenient for tourists, so bottled water is a good option unless you plan on going off the beaten path.
You can find bottled water in most restaurants, hotels, big/small groceries, etc. Many hotels offer a couple of free bottles per guest each day.
You can also buy 19-20L jugs of bottled water for not much money, and use it to refill smaller bottles as needed. Cheaper and more convenient than having to buy a bunch of small bottles every day.
Many locals drink hot tea instead of water for meals. The water is boiled, so all the critters in it are no longer a risk.
Avoid ice unless you made it yourself, since it was likely made from tap water. If the restaurant uses ice delivered from outside companies rather than making it themselves, it's safe -- but use good judgement.
Instead of a filter or chemical treatment, use UV sterilization products like Zippz posted. This one sterilizes 50 Liters of water on 4 alkaline batteries, or 150 liters on 4 lithium batteries. Takes about 2 minutes per treatment, and it's good for about 3,000 treatments.
https://www.steripen.com/Aqua