Pipelines in DataBrew are powerful features that allow you to aggregate and process data from multiple sources in real-time. They serve as a central hub for data flow and event processing.

​
Key Aspects of DataBrew Pipelines:

  1. Data Aggregation: Combine data from multiple connectors to create a unified data stream. Can also work with a single connector if desired.
  2. Real-time Event Processing: Receive instant updates and events from connected systems as they occur.
  3. Unique Identifier: Each pipeline is assigned a public pipeline ID upon creation.
  4. SDK Integration: Use the public pipeline ID with DataBrew’s SDK to establish connections and receive live updates.
  5. Flexibility: Configure pipelines with one or more connectors for simple or complex data flow setups.
  6. Event-driven Architecture: Facilitate an event-driven approach, enabling applications to react immediately to changes in data sources.

By utilizing pipelines, you can streamline your data processes, enabling real-time data integration and analysis across multiple platforms within your DataBrew environment.

Was this page helpful?

ConnectorsSecurity in DataBrew