Roughly, “to screw up one eye” means to tighten and tense up all the muscles surrounding that eye. This is an image of someone screwing up their entire face. Imagine that, but more localized. And you probably wouldn’t close the eye completely, but rather leave a narrow gap remaining to look disprovingly at the person with. Basically an extreme squint. Here is an image of someone screwing up …
