At home my M & P 40 is always loaded and close by, but I'm not comfortable with a round in the chamber.
Some people here would laugh at you, but it's really your choice. If you feel uncomfortable, then don't chamber a round.
A lot of people do not feel comfortable around guns. Nobody here would criticize those people for not having a sd gun.
At least you took the initiative to (presumably) take a safety course and invest in a fine firearm for self defense.
If I were you, I would practice loading and firing, as you would have to do in an emergency. I suppose there is some deterrent effect of the sound of a slide racking a round into the chamber, like the distinctive sound of a pump shotgun shucking a round into the chamber.
I presume your pistol is secured into a quick access lock box of some sort.
Personally, I do keep a round chambered in my SD gun.