You never mention the most important part. Who do the children live with?
That is the key to who pays who child support.

If they live with you then he pays. Decent money or not, it's his responsibility. They will tally his income,and come up with a monthly amount he HAS to pay. If he makes little then they will give him what they think he can make. The court determined that I was able to make a certain amount, even though I don't make that amount the court felt that I could so they used it to determine my income.

If the children live with him then you pay, even though you were decent about not making him pay. If they live with him it is your responsibility to pay. Sorry to say the courts are not going to care one way or another about your school loans. I know it sounds harsh ( and I don't mean it to) but it's the truth.

If the kids now live with him, was your agreement that he not pay a legal one or just verbal. Legally he owed you that money so you may be able to use that against what he wants from you now.