Yes, unless you cannot conceal it any other way, I would avoid them. They are uncomfortable, you can't take off your jacket or coat in public, and a loaded weapon will likely be pointing at someone anytime you are around people.
In addition, they need to be tied down to your belt to keep them close to the body which increases the discomfort, though may help to hold up your pants.
And finally, unless you are carrying two pistols the load is unbalanced. Even with two loaded mags on the off-side.