What type of ammo are you using? What length barrel? What choke? What gauge shotgun?
Birdshot out of a short-barreled .410 probably wouldn't kill a person at the far end of your house. Buckshot from a 12ga magnum shell could easily penetrate several layers of drywall and be lethal, or kill a person 40 yards away. A rifle slug from a 12 ga loses accuracy quickly, but has easily enough power to kill a person at 100 yards. Sabot slugs thru a rifled barrel from a 12 or 20 ga rival the performance of .30 caliber rifles.
It's all about mass and velocity, really. The larger the projectile is, the more momentum it retains, the more material it can penetrate, and the farther it can travel. Bird shot or game shot, relatively small pellets, are quite lethal when theya re massed together. As they spread out, and that mass becomes numerous tiny projectiles, the effectiveness falls off pretty quickly. However, keep in mind that a 12 ga with bird shot kills birds, even large ones, pretty easily at 30 yards.
Keep in mind shotguns are the victims of a LOT of bad press and myths. You still need to aim. You still need to hit the target in an effective spot. Missing is still bad.