Obsesrvability with ETCD
Explaining how to use ETCD for managing pipeline statuses in Blink.
Introduction
Blink has an ability to use ETCD server to register pipelines. It allows to you to see if pipeline if running or not. In this article, we will walk you through how to use ETCD for managing pipeline statuses in Blink.
How does it work
If ETCD configuration is provided - Blink will register pipeline in ETCD server. It will create a key with the pipeline id and value with the pipeline status. The key will be updated every time the pipeline status changes.
Blink sets the key with cetrain status for a defined TTL. If the TTL is expired - the key will be removed from ETCD server and you can reveice a notification about it.
Format of the key in ETCD server. PIPELINE_ID will be replaced with the pipeline id from the configuration file.
Default TTL is 15 seconds. Blink will try to refresh the key every 10 seconds.
Statuses
Starting ServiceState = “starting” Loaded ServiceState = “loaded” Started ServiceState = “started”
starting status is being sent to ETCD server when the pipeline is starting. Normally you will see it once. If you receive it multiple times within a short period of time - it means that the pipeline is restarting.
loaded is a transient status. It means that the pipeline is loaded and ready to start. It will be followed by started status.
started is a final status. It means that the pipeline is running.