Your custody agreement should be very specific about this. If he is late returning her you need to be firm about him returning her on time. He is in contempt of court keeping her longer.

Do whatever you have to including sending the police to his house with a copy of your divorce papers stating his allowed time.

If you let him get away with this now it will only give him a reason to think you are going to cave in to him in the future. Nip it in the bud now and let him know you only have your child's best interest in mind.

I suspect he loves his daughter. He's obligated to return her of course. But if the custody arrangements are pretty unequal, can you consider allowing your daughter to stay longer with him in the future when school isn't in session? He might be more cooperative. If you keep it informal there is no need to go to court.