If you plan on being a thief and pickpocket, you’ll have problems doing so if you’re so radiantly beautiful that people just follow you around all the time and rain gifts on you in the hope of getting your attention. If you plan on being a social butterfly, you’re going to have a hard time of it with extremely low charisma.
However, you do need to be aware of it, as it will impact how well you can use certain features of the game. You don’t have to max or min your charisma. Low Charisma, however, means that people don’t like you, but also don’t pay attention to you, so you can use your lack of social presence to pickpocket items from others. High Charisma means you’ll be the center of attention, people will casually go up to you and invite you to join sects, give you gifts, and ask to spar or train with you.
There are benefits to both high and low charisma. Now, in this game, how your character looks is going to directly impact their charisma, and by that metric how they’re interacted with by others.