Add metrics on metric probing

[liu-cong]

• Latency: Low metric fetch latency is crucial as the load balancing algorithm relies on near real time metrics from the model servers. Therefore it's important to monitor the metric fetch latency.
  Currently we simply log the latency, we should integrate Prometheus and add a Histogram metric.
• Error count. This can help detect bad backends.

Other consideration: We may need to sample the metrics instead of recording every probing call, due to the high probing frequency.

[MadhavJivrajani]

I'd like to try this out
/assign

[kfswain]

@JeffLuoo just an FYI