Skip to main content

Operational Hints & Tips

External Deletion of Object#

If a SOLE managed Snowflake object is deleted externally, it can lead to error during execution of pipelines.
As external deletion do not update the state file of the resource, SOLE assumes that object still exists.
This leads to error when SOLE tries to refresh the state of the object but is unable to find the object.

To handle such scenarios environment variable LIFECYCLE_STATE_RESET can be used to reset the state file for Resource Group to which the Object belongs to.

Implicit Deletion of Database-Level Objects#

A Database managed by SOLE can be deleted in one of the following ways(via Pipeline):

  1. Running Delete Jobs in Feature and Dev branches
  2. Deletion of Database by removing definition from YAML files

If Database-Level Objects are not deleted by SOLE(by delete job), but implicitly(by Database deletion), it can lead to failure in Pipeline.

To handle such scenarios environment variable LIFECYCLE_STATE_RESET can be used to reset the state file for Database-Level Jobs.

Privilege Revokes on Objects shared in multiple Environment#

If a managed resource is shared between environments(no environment specific suffix added, see namespacing), then grants on the object would be reset in each pipeline as per the environment.

roles:  ROLE_1:    namespacing: suffix    comment: Role 1warehouses:  WAREHOUSE_1:    comment: Warehouse 1    size: MEDIUM    namespacing: prefix    grants:      MONITOR:        - ROLE_1

In the above example, one role and one warehouse is managed by SOLE.
The role ROLE_1 is created in each environment, whereas the warehouse WAREHOUSE_1 is created once and shared between environments.

If pipeline runs in PROD environment, then MONITOR on WAREHOUSE_1 would be granted to ROLE_1_PROD.
When the same pipeline runs in QA environment, the grant to ROLE_1_PROD is revoked and MONITOR is granted to ROLE_1_QA.

Resource Monitor Updation#

Add bug link [admin] Due to a bug in Snowflake, a resource monitor cannot be created with a complete timestamp(YYYY-MM-DDTHH:MM:SS format) in parameters start_timestamp or end_timestamp. The parameters only support date(YYYY-MM-DD) at the time of creation a Resource Monitor.

As only date can be provided to create Resource Monitor, the value of both start_timestamp and end_timestamp parameters must be a valid date(greater than current date).
After creation, the local state gets updated for the Resource Monitor. As Snowflake converts the date to a complete timestamp, the state has timestamp as value for the parameters.
Due to this, when pipeline is run again, the resource monitor is updated as configuration(date) differs from value in state(timestamp).

To remove the updation, the value of the parameters start_timestamp and end_timestamp should be updated to complete timestamp after creation of the resource monitor.

Warning Message in Destroy-Plan and Destroy Jobs#

After generation of destroy-plan or deletion of resources, terraform might log Warning messages such as below:

│ Warning: Resource targeting is in effect│ │ You are creating a plan with the -target option, which means that the│ result of this plan may not represent all of the changes requested by the│ current configuration.│       │ The -target option is not for routine use, and is provided only for│ exceptional situations such as recovering from errors or mistakes, or when│ Terraform specifically suggests to use it as part of an error message.
│ Warning: Applied changes may be incomplete│ │ The plan was created with the -target option in effect, so some changes│ requested in the configuration may have been ignored and the output values│ may not be fully updated. Run the following command to verify that no other│ changes are pending:│     terraform plan│   │ Note that the -target option is not suitable for routine use, and is│ provided only for exceptional situations such as recovering from errors or│ mistakes, or when Terraform specifically suggests to use it as part of an│ error message.

These warning messages are generated when not all the Objects in the Resource Group are deleted.

If an Account-Level object is not namespaced(namespacing set to value other than both[default]), it would not be deleted when delete job runs.

This is to ensure that objects shared by environments are not deleted by feature-branches.

Last updated on