FIRE your lawyer she is not worth the paper her degree is printed on. Get a new lawyer, get free aid anything is better than a lawyer telling you "good luck" ( I'm on lawyer #3 now) File for custody modification. You have the right to do so at any time during the life of your child. Be prepared. DO NOT go into the hearing with the reasoning that you just gave here though. A judge will not except "how can I raise a baby when I don't know her schedule, her diet or her milestones" as a good enough reason to change the 50/50 custody you have now. As mothers we all know that it is hard and unfair but a judge will see you as trying to control and take away time from the father. You will be made to look like a fool.
Prove that he is not taking care of the child. But remember that in the eyes of the court his time is his time and he can spend it as he wants.
Can you provide better care for the child while he is at work? Does your child require a special diet that is health related and they are not providing that? Are they neglecting your child? Has he hurt the child? These are things that can help you. Please don't make up things though, it will only hurt you and your child. If you can prove that he is neglectful or that his mother is neglectful then you can try for full physical custody( maybe even full legal custody) with visitation. If you get that you are entitled to child support.
I made "special rules" that my ex had to agree to when he has our kids. He is not able to drink or have anything to drink 24 hours prior to our boys coming to his house, his GF is NOT allowed to be alone with our boys, he has to be there when he has the boys, no babysitters. I insisted it be written in our orders. After all why should he get visitation and not be there when they could be home with me.
I hope all works out for you, good luck. ( I can say that because I'm not hired to defend you )