Snowflake Migration Best Practices

Snowflake Migration Best Practice #1:

Ensure Your Technology Stack has the Following Features

  1. End-to-end encrypted connections:
    Data Security teams should secure all connections between on-premises data sources and the Snowflake data cloud with end-to-end encryption. This is important to prevent data leakage and misuse during the migration process.
  2. Dynamic Sensitive Data Masking:
    Ensure that your technology stack has dynamic data masking features to ensure that sensitive data is masked while in transit. This serves as an additional layer of security for sensitive data from unauthorized access.
  3. Data Cleansing:
    Select a user-friendly solution that can cleanse data efficiently, and ensure it is valid and complete before migrating the Data to Snowflake. Effective data cleansing means high data quality.
  4. Data Catalogs:
    Ensure there is a solution that can maintain automated data catalogs of all activities performed during the migration process. The solution should have a continuous, scalable, and auditable data flow for analytics.

Snowflake Migration Best Practice #2:

Plan for the following Key Requirements before starting the Snowflake Data Migration Process

  1. Determine Data Storage Requirements:
    Estimate the amount of data/storage and time it may take to migrate. If the storage is more than 50 TB and time is short, consider using physical storage devices to transfer large amounts of data.
  2. Determine your Network’s Speed:
    Determine the bandwidth and connectivity available between your on-premises server to Snowflake (e.g., Direct connect, Region/location of the source and target, etc.). This will determine how much time the actual data migration will take.
  3. Determine Role-based Data Access needs:
    Discuss data access needs to understand who will be using this data, the access frequency, and how fast they want to access.
  4. Set Achievable Timelines:
    All of the factors above contribute to setting achievable timelines for migration. For example, there might be fixed deadlines to offload data from the on-premises database. Tight deadlines complicate the Snowflake data migration process as unforeseen problems (e.g. network breakdowns, equipment malfunctions, etc.) might impact the project’s timelines. It is advisable to keep a buffer when you are planning timelines.
  5. Use the new ELT approach to data migration:
    ELT refers to “Extract, Load, Transform,” and is a modern variation on the older process of “Extract, Transform, and Load (ETL)”. ETL runs transformations before the data is loaded to the data cloud, resulting in a more complex, lengthy, and expensive migration process.On the other hand, ELT transforms data after it is loaded to the data cloud. This means that organizations can transform their raw data at any time, when and as necessary, streamlining the data loading process and saving resources. ELT is beneficial for cloud-native data warehouses like Snowflake because data transformation happens within the target destination itself.

Snowflake Migration Best Practice #3:

Plan and Manage Costs Effectively

  • Which roles should have access to Snowflake, what privileges they have, and why they need them?
  • Your data governance policies can help answer the access and privileges part of this question. To understand why users need access and other privileges, you need to dive deep into their roles and responsibilities. Ensure that access is granted only to users who absolutely need it and understand how the per-query pricing model works.
  • What are the typical data workflows, data usage scenarios, and storage/compute requirements?
  • Snowflake invoices its customers only for what storage and computing power they use. For instance, Snowflake storage costs can begin at a flat rate of $23/TB/month. Compute costs start from $0.00056 per second, per credit, for the On-Demand Standard Edition. So, it is crucial to determine this part to control costs.
  • Which data must be moved to Snowflake, and which data should remain on-premises?
  • Efficiently balancing data storage between on-premises and Snowflake will help optimize your cost structure even more.

Secure Snowflake Data Migration with Securiti

Data Governance for Snowflake

  • Dynamic Data masking based on roles and policies to restrict access & usage of sensitive data from unauthorized personnel.
  • Table, column, and even row-level access policy enforcement.
  • User access history audit to detect any non-compliance with governance policies.

Data Privacy for Snowflake

  • Data Mapping and Classification of personal data.
  • Quick and accurate DSR fulfillment.
  • Using a conversational interface (Auti) you can extract any individual’s personal data within minutes.
  • Comprehensive Privacy Risk Assessments that enable a proactive approach to risk mitigation.
  • Data Breach Management Notifications that meet strict regulatory requirements and notify all impacted parties as quickly as possible.
  • A Workflow Orchestration feature that uses a simple drag-and-drop design and helps automate various privacy, governance, and security functions within Snowflake.

Data Security for Snowflake

  • Network Security:
  • Site access is controlled through IP allow and block lists, managed through network policies.
  • Account/user authentication:
  • MFA (multi-factor authentication) for users’ increased security for account access.
  • Automated security scanning of any misconfigurations. Snowflake Security Administrators can decide to remediate any misconfigurations automatically or receive notifications.
  • Compliance with Data Regulations like PCI-DSS, HIPAA, and more.
  • Map security policies to specific standard controls and regulatory compliance.
  • Generate one-click reports to demonstrate compliance coverage to regulators and auditors for various data privacy and security regulations.

Data Governance:

--

--

--

All Thing Data Privacy & Security

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Spring Framework for Beginners — Part 1

Assisting You Pick the Right Memory FoamMattress https://t.co/pCiUKeY5UT

What Is ‘Multiple Strategy’ Farming?

SYNERZIP’S OFFICE PRODUCTIVITY SPACE EXPERTISE

Service Locators: Anti-Pattern or Just Misunderstood

Everything About Python List Data Structure: Beginner’s Guide — PyShark

Making 2048 Game in Flutter by using Explicit Animations — Part 5

Does your Company need Data-Driven Testing

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Privacy Research Team, Securiti

Privacy Research Team, Securiti

All Thing Data Privacy & Security

More from Medium

Four Avoidance Strategies for Improving Cluster Resilience, Performance and Outcomes

How do AWS Leadership Principles Help in the Organizational Growth

Microsoft Teams as a UCaaS platform: An effective way to use Teams for Internal as well as External…

Microsoft Teams. UCaaS. Unified Communications.

Microsoft Azure Fundamentals Training Series | 3-Azure Service Models