emotional tug-of-war

I've been back to work for two months now (hence why I don't post as often as I used to). I'm getting more and more comfortable with leaving baby J at home with grandma. Being back at work definitely makes me feel like I've gotten a little more of my identity back. But it is NOT easy by any means...
When I get home I hear stories of all his new found talents, and all the milestones he's reaching. I am excited that my baby is growing and learning so much, but I'm sad because I'm missing so much of it! I can't help but feel a little guilty for not being there... I'm supposed to be his biggest fan, his cheerleader for life...

But how do I do that and still have my own life? I leave home usually before he wakes up, and I get home just before dinner time. When I get home, he has all my attention through dinner time, shower time, and bed time. This generally takes about 2 to 3 hours (after working 12 hours). When he goes to sleep my hubby and I try to catch up on how our day was over a glass (or two) of wine.
So I basically have no "me time" except when I take a break at work. Yet I still feel guilty for leaving baby J at home... I feel like I'm missing out...
Will he remember that I wasn't there? Will he understand that mommy had to work to provide for necessities? Will he care?