Which solution will fix this problem?
Create an Amazon Route 53 Resolver inbound endpoint. Add a forwarding rule for the domain example.com. Associate the forwarding rule with the VPC.
Create an Amazon Route 53 Resolver inbound endpoint. Add a system rule for the domain example.com. Associate the system rule with the VPC.
Create an Amazon Route 53 Resolver outbound endpoint. Add a forwarding rule for the domain example.com. Associate the forwarding rule with the VPC.
Create an Amazon Route 53 Resolver outbound endpoint. Add a system rule for the domain example.com. Associate the system rule with the VPC.
Explanations:
An inbound endpoint allows DNS queries from on-premises systems to be routed to Route 53. However, for the EC2 instance to resolve a DNS name from on-premises, an outbound endpoint is needed, not an inbound one.
A system rule for Route 53 Resolver is used for managing DNS resolution settings, but it is not the correct configuration for forwarding queries from the VPC to an on-premises DNS server.
An outbound endpoint allows the EC2 instance in the VPC to forward DNS queries to an external DNS server (in this case, the on-premises DNS server), and a forwarding rule for the domain example.com ensures that queries for that domain are sent to the on-premises DNS server.
A system rule applies to internal DNS resolution, and it would not work for forwarding queries from the VPC to an on-premises DNS server. A forwarding rule is required instead.
To the best of my knowledge, the answer is:
Create an Amazon Route 53 Resolver outbound endpoint. Add a forwarding rule for the domain example.com. Associate the forwarding rule with the VPC.