Too much eye contact can be threatening (predatory animals are really good at staring). There’s a fine line between eye contact with romantic/sexual overtones and aggressive/scary/creepy eye contact. If they’re avoiding your gaze then it might be the latter.