It is important to understand that any TCP packet that is routed over a network requires bi-directional support. This means that even though I may have a route on my router to your network, if you don’t have a route back to me, the transmission of packets will fail. TCP uses acknowledgments to confirm the packet has been received, and if the acknowledgment cannot be routed back to the source the traffic will not continue to flow.
The same applies with protocols such as ping. Ping sends a echo to the destination, and when the destination device receives the ICMP packet it must send a echo-reply packet back to the source. If there is no route going back to the source, ping will be unsuccessful.
In the live demo below I have illustrated this exact problem. On my router AOIP.ORG I have tried to ping the IP address 126.96.36.199 (the loopback interface of R2). AOIP.ORG has a default gateway of last resort configured to send all traffic to the router R1 (R1 already has routing configured correctly), R1 will send the packet to R2 however R2 does not have a route back to the 192.168.1.0 network (attached to AOIP.ORG). Only after I telnet into R1, and then telnet into R2 and configure a route to the network 192.168.1.0 do my pings begin to work. ( I can’t telnet directly to R2 since it does not have a route back to me, so I have to do this on a hop by hop basis).