Probe in kubernetes

There’s two kinds of probe: readinessProbe, livenessProbe in kubernetes used to detect if your service is healthy.

We encountered a problem when configured readinessProbe, there’s a property named initialDelaySeconds which indicate kubernetes will start health check after specific second, we used the default value 60 which means kubernetes will check health after 60 seconds.

readinessProbe:
  initialDelaySeconds: 60
  timeoutSeconds: 5

As we deployed over 20 StatefulSet pods and these pods joined as a cluster which cost over 60 seconds, kubernetes can’t ping service successfully so that kubernetes restart these pods, thees pods restart in loop all the time.

After we increased the initialDelaySeconds to 120, everything goes fine.