Yes, but it seemed that baby was going to be a story of significance somewhere.
I understand the thought he shattered when the NK did, but if he wasn’t there then why?
That baby was just one of many that the NK had gotten from Craster. He was breeding his daughters, he kept the girl babies while the NK got all the males in return I assume for not killing them. We saw the NK turn the baby and according to the show if you kill the one that creates them, they also die so it's all but assured that those babies are dead too wherever they were. I'd say they were meant to be white walkers in his future army.