Consume Consume

yaml
type: "io.kestra.plugin.gcp.pubsub.Consume"

Consume messages from a Pub/Sub topic.

Requires a maxDuration or a maxRecords.

Examples

yaml
id: gcp_pubsub_consume
namespace: company.team

tasks:
  - id: consume
    type: io.kestra.plugin.gcp.pubsub.Consume
    topic: topic-test
    maxRecords: 10
    projectId: {{ secret('GCP_PROJECT_ID') }}
    subscription: my-subscription

Properties

serdeType

  • Type: string
  • Dynamic:
  • Required: ✔️
  • Default: STRING
  • Possible Values:
    • STRING
    • JSON

The serializer/deserializer to use.

subscription

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️

The Pub/Sub subscription.

The Pub/Sub subscription. It will be created automatically if it didn't exist and 'autoCreateSubscription' is enabled.

topic

  • Type: string
  • Dynamic: ✔️
  • Required: ✔️

The Pub/Sub topic

The Pub/Sub topic. It must be created before executing the task.

autoCreateSubscription

  • Type: boolean
  • Dynamic:
  • Required:
  • Default: true

Whether the Pub/Sub subscription should be created if not exists.

maxDuration

  • Type: string
  • Dynamic:
  • Required:
  • Format: duration

Max duration in the Duration ISO format, after that the task will end.

maxRecords

  • Type: integer
  • Dynamic:
  • Required:

Max number of records, when reached the task will end.

projectId

  • Type: string
  • Dynamic: ✔️
  • Required:

The GCP project ID.

scopes

  • Type: array
  • SubType: string
  • Dynamic: ✔️
  • Required:
  • Default: [https://www.googleapis.com/auth/cloud-platform]

The GCP scopes to be used.

serviceAccount

  • Type: string
  • Dynamic: ✔️
  • Required:

The GCP service account key.

Outputs

count

  • Type: integer
  • Required:

Number of consumed rows.

uri

  • Type: string
  • Required:
  • Format: uri

File URI containing consumed messages.

Definitions

Was this page helpful?