Will replacing my air filter really improve heating efficiency?

Yes. A clean filter improves airflow, reduces strain on your system, and helps it heat your home more effectively.