Thanks for the information. It is definitely a firewall issue. I can connect just fine with the firewall off, but when I have it on then I get the errors.
The firewall rules appeared to be correct, but something is still configured wrong. At least I can connect and keep troubleshooting the rules.