ICMP ECHO-REPLY packets are the responses sent back from the host that is being PING'ed. ICMP ECHO-REPLY messages are actually encapsulated within an IP datagram even though the standards document for IP states that IP itself must provide the ICMP functionality. Always remember that the ICMP ECHO-REPLY is a different datagram from the ICMP ECHO request and it takes additional clock ticks to copy the ECHO request information, destroy the original ECHO request, create a new ECHO-REPLY, encapsulate it within IP, CRC check the IP headers and then transmit the reply. This of course assumes that the transmit and receive buffers on the receiving host are empty and the interfaces are ready to transmit and receive. If the hardware is not ready, additional delays will occcur.
All of this adds to latency that is not the network's fault. Don't blame your ISP if you get slow PING response times or no response from a very busy server or your own Internet gateway or link is flooded with traffic.