Talk to him. Relationships require communication. Just say, “Hey, it made me feel kind of weird when you made that joke the other day. Do you actually flirt with other girls?”
Also, even if he does look at other girls, that has nothing to do with you and everything to do with the fact that he is a human and his relationship status does not make him dead. I’m in a relationship and I definitely check out other people. Does it mean I’m unhappy in my current relationship? Nope! Not in the slightest. What it does mean is that I am alive, I am a sexual being, and I have eyes.