Confused about relationship, incompatible after all?

I was in a ltr/married first 14 years, 2 years divorced now with two small children, 50/50 custody.

I've been dating a guy for nearly 9 months now. I've had a nagging feeling since the beginning that long term wise we wouldn't be suitable however I wasn't looking for anything serious and still dealing with the fallout of my divorce when we met. My boyfriend has had a handful of relationships ranging from 3 months to 3 years however he's never lived with anyone, no children. In the beginning we had lots of fun dates and were seeing each other whenever we could, some weeks that was 3 times or so, I started to get stronger feelings and open up more etc, I was relieved to find he's a great communicator. Over time things started to get more comfortable and the fun days became just spending time at my house cooking dinner, watching netflix, cuddles etc. There was a few month period where he was staying over semi regularly when my kids weren't here but then one night he decided to leave 10pm or so instead, just saying he wanted to go home. He never stayed over again. I didn't push it but I did feel hurt as I enjoy the closeness and intimacy of staying the night together and he has a roomates so staying at his isn't ideal.

Since he's been single so much hes used to just doing as he pleases which is fine. As times progressed I've found i want to see him more but lately when our free time aligns, he'll choose to see friends, hobbies, relax at home etc and whilst I wouldn't want to be with someone who has nothing else but me I feel hurt as I only have so much kid free time and if he chooses to do other things (nothing I'm invited to or included in) then we can miss each other for weeks. I asked him how he feels about not seeing me for 2 weeks and he said it doesn't really bother him as we talk everyday. I feel like because it bothers me I'm obviously more into him than he is into me as I don't want to make a habit of seeing him once a fortnight! We had gone months of seeing each other every Monday night and then all of a sudden he said he's actually just joined an indoor sport team and is now unavailable. That's great and all but that leaves me thinking he gave up our one guranteed night and literally didn't tell me until I asked "what should we do tomorrow" and he said he forgot to tell me he can't do mondays anymore.

I'm just drained, in amongst all this as i thought things were going great and we seemed serious, I introduced him briefly to my children and he'll talk about seeing them more but then it's like he'll work out his schedule first and see if he has better offers. At 9 months I often feel like we are going backwards not forwards, I'm not asking him to be a stepdad, or move in or anything but I have enough availability and love to give someone something more meaningful.

Is it unrealistic to think I could have a partner that sees me a few times a week and can stay over regularly or vice versa after 9 months?