Is it wrong to deny my husband sometimes while I'm pregnant?

Im nearly 6 months pregnant with our 2nd child. This time around Ive been more tired and more aches and pains than usual. My husband asks fo everyday.... sometimes at 5am while I'm trying to catch up on some sleep. When I say no, he says "You can just lay there." When I succeed in denying him , he pouts and makes me feel bad about it until we actually have . He says he doesnt make me feel bad but he does, trust me. How can I get about explaining to him that we don't need to have everyday while I'm pregnant and still be happy together?