However, if you are a stable adult, the grandparents have very little chance of removing custody from you. In fact, if the instability of the father is the issue, you could petition for full custody.
The biological parent trumps the grandparents. They are legal strangers to the child. They also have no responsibility for paying child support. You need to be prepared to raise your child on your own.