Their job is to make money

Waiving the closing costs cuts into their profit. Providing the appraisal cuts into their profit. A small town bank probably doesn’t make enough money to give away perks like that.
Rolling the closing costs INTO the loan means the loan is higher by $2500, which the house won’t support. Regardless of their fees, it sounds like thay can’t give more than about 70% Loan-to-Value(86K is about 68% of 126K). THAT is your deal-killer. I’m in a similar situation, although a bit worse because I’m underwater. My loan is an ARM with a balance of $101,500. I’d love to refi to a fixed rate, but the house is worth less than $80K. So the property won’t support the loan that I need.
In your case, you’d have to find a different bank that would support a higher LTV – you’re looking at about 90%. Problem is, such a loan would have ridiculous interest rates, so you’d essentially be right back where you started, AND be out the appraisal and closing costs…