On that same basis though, you can consider shooting a right-handed weapon as a left-handed shooter as "unsafe" based on conflicting design principles. Or a right-hander to shoot his right-handed handgun in a left-handed stance. I agree with you for the most part though, if it's not designed to do it, chances are there are safer ways to do it.
We can consider writing a poem safer than the act of shooting a gun, or popping a wheelie, or jumping off a plane, or driving, or swimming in the ocean for that matter. So when we leave the realm of seemingly invulnerable safety, it's down to the person's training and experience as to what level of "safe" he is currently capable of.
Don't get me wrong....I can't see a single damned reason why you would ever want to shoot your handgun upside down. Maybe you horrible twist your arm, but your nerves are still viable and you're able to grip and fire your gun, and you're training for that incident...

But I cannot see that as being "unsafe" as a rule.
Just whatever change is left in my pocket!