I wish I knew the answer to your question from a legal aspect. But from one mother to another -- I don't think you were wrong at all.
My own ex does not understand that our son has Autism/ADHD. He says he understands what this means, and has all this experience dealing with Autism, but he doesn't care at all. It's a point of contention for us both -- and we fight often -- our divorce was a nightmare.
That said, I believe there is a point where, when it all goes away -- all the lawyers, and the GAL, and the counselors -- where you are just stuck. You and your child. I believe we as mothers have the responsibility to do what we believe with our whole heart to be best for our children, regardless of the cost.
I would say don't worry about it unless he takes you to court. Make HIM do the talking. And if he does, then you stand up in court and explain the situation.
Courts cant be there 24/7. At some point, we have to use the judgement we came into this world holding, and say the heck with the rest.
Keep doing what is best for your baby. The rest is just white noise, in my opinion.