Collar is too important when it comes to shirt, one should be able to comfortably fit two fingers inside your buttoned collar without it tightening against your neck, if turning your head, causes the collar to turn with it, then the collar is too tight. And if there is a lot of room around your neck, after putting up the collar button, then the collar is very loose.
Many people set the shoulder fit wrong. A shirt with its seam sitting on your shoulder bone is the fit to go for.
The body part of the shirt should give you a relax, at the same time draped fit. A perfect fit will give you enough comfort around your armpit and will not reveal your body shape, when you sit after a meal.
If one wear's a full sleeve shirt, perfect fitting cuffs are the integral part for comfort. Cuffs should fit a bit looser than a watch and fall two centimeters from your wrist bone.
If you can see the creases on your sleeves when you move your hand they are too tight for you, but they also should not be so loose that they hang down your arms. The perfect sleeve will let you bend your arm, without moving your cuff more than an inch from your wrist.