Skip to main content

Stage

Configuration can be provided to Snowflake Object Lifecycle Engine for the following operation with stage:

  • Manage Lifecycle of new and existing Stage
  • Manage Grants of Stage

Supported Parameters#

The runner supports below parameters.

  • URL: Specifies the URL for the stage.
  • CREDENTIALS: Specifies the credentials for the stage, required only if the storage location is private/protected; not required for public buckets/containers.
    • Configuration key: credentials
    • Data Type: String
  • COPY_OPTIONS: Specifies one (or more) copy options for the stage.
    • Configuration key: copy_options
    • Data Type: String . See here for copy_options parameters.

      See Snowflake Docs section for copy options.

  • ENCRYPTION: Required only for loading from/unloading into encrypted files; not required if storage location and files are unencrypted.
    • Configuration key: encryption
    • Data Type: String
  • FILE_FORMAT: Specifies the file format for the stage.
    • Configuration key: file_format
    • Data Type: String/Map. See [here](/docs/sole/objects/copy_into_copy_options#File Format) for file format parameters

      See Snowflake Docs section for file format .

  • STORAGE_INTEGRATION: Specifies the name of the storage integration used to delegate authentication responsibility for external cloud storage to a Snowflake identity and access management (IAM) entity.
  • COMMENT: Specifies a comment for the stage.
    • Configuration key: comment
    • Data Type: String
  • MANAGE_MODE: Configures what properties to manage for the Stage.
    • Configuration key: manage_mode
    • Data Type: String
    • Possible Values:
      • none
      • grants
      • all(Default)
  • GRANTS: List of Privileges and Roles to which privileges are granted to on the current Stage.
    • Configuration key: grants
    • Data Type: Map

Storage Integration Usage#

A Storage Integration can be specified in an external stage if Access is managed via Storage Integration.

In the configuration of Storage Integration, paths to allow external storage can be specified as well.
If such paths have been specified, then the stage using the Storage Integration must have the value of parameter URL as one of the allowed paths.

In the event, both Storage Integration and Stage are being updated, an error could occur due to invalid URL parameter.
To resolve such error, follow the following procedure:

  • Disable Stage(commenting the definition in configuration) and its dependencies and run a pipeline. This would delete the stage from the environment.
  • Enable the stage(uncomment definition) with updated configuration and run pipeline again to initialize the stage from scratch.

Supported Stage Grants to Roles#

Following is the list of Privileges Grant to Roles that can be specified in the stage definition

  • ALL PRIVILEGES
  • USAGE
    • Only for External Stage only
  • READ
    • Only for Internal Stage only
  • WRITE
    • Only for Internal Stage only

Implicit READ for Internal Stage#

If Only WRITE is granted to a Role, READ privilege is implicitly added as well.

Basic syntax#

stages:  <stage-name>:    <configuration-key>: <value>

Examples#

databases:  SALES_RECORD:    comment: "product sales record"    schemas:      comment: "sales records for year 2021"      stages:        S3_STAGE:          comment: "files stage"          url: "s3://<my-bucket>/<public-file>"
Last updated on