Introduction

Blink configuration file is a YAML file that specifies the behavior of the Blink service. The configuration file is used to define the source and destination of the data, the behavior of the service, and the secrets used in the service.

Configuration file structure

Configuration file for Blink can be divided into the following sections:

  • Service
  • Source
  • Secrets
  • Processors
  • Sink

We will go through each section in detail.

Service

Service is a top-level section in the configuration file. It is used to define the basic configuration of the Blink’s pipeline, metrics, offset storage and unique identifier of the pipeline which will be exposed in the metrics.

service
Service Object

Service is a top-level section that contains the following nested fields:

Source

Source section specifies the source of the data. It can be a file, a database, a message queue, etc. Pretty much any driver supported by Blink.

source
Source Object

Service is a top-level section that contains the following nested fields: