It must be seen that you aren’t a being that smiles.
Gaze is a contemplative gesture. Downward gaze might represent a self remembering what it has felt; upward gaze might represent a self imagining what it is going to feel. In these instances and all others, the represented self is a self looking inward. Nothing outside the self can be thought to matter. That is, in a portrait nothing except the self can be seen to matter.
Technical note about that: in a portrait, selective focus has the effect of subordinating and excluding from consideration all that is not face, contemplating.
Test: as portrayed, does your face entail the term “distinguished”?