Data Warehouse System Requirements

Hardware and software requirements for cloud, hybrid, and on-premises data warehouses/data marts.

DataSelf Cloud Data Warehouses

There are no system requirements. DataSelf will securely host your data warehouse(s) in our AWS or Azure servers.

MS SQL Server for On-Premises or Private-Cloud Data Warehouses

Please use the following guidelines when hosting your data warehouse on your own MS SQL Server:

MS SQL Server versions: 2022, 2019, 2017, 2016, or 2014 Azure SQL.
MS SQL architecture:
- Server or serverless, on-premises or cloud.
- Azure SQL (click here for more info).
MS SQL Editions: any.
SQL authentication supported: Windows, SQL, Azure Entra ID Password.
Guidelines for hardware specs:
- Data space (HD or cloud space): In most deployments, the data warehouse data space will be 5% to 20% of the source database data space (source’s data space without logs). For server-based SQL, we recommend using SSD with at least 150GB free or 5+ times the expected data warehouse data space, whichever is the largest.
- RAM: For fast performance, allocate SQL to have at least twice as much RAM as the size of the largest table to be hosted in the data warehouse. For instance, allocate at least 16GB of RAM for SQL if the largest data warehouse table will have 8GB of data. This will dramatically reduce the risk of SQL needing to use HD swaps when working with your large tables.
  - For top performance: all the data warehouse data space should fit into the available RAM.
- CPU and RAM: Here are guidelines based on the expected data warehouse data space:
  - Up to 4GB: 2+ CPU threads, 2GB+ of RAM. Azure SQL eDTUs: 100 to 200
  - Up to 16GB: 4+ CPU threads, 8GB+ of RAM. Azure SQL eDTUs: 200 to 300
  - Up to 32GB: 4+ CPU threads, 16GB+ of RAM. Azure SQL eDTUs: 300 to 800
  - Up to 128GB: 8+ CPU threads, 64GB+ of RAM. Azure SQL eDTUs: 400 to 1,600
  - Up to 1TB: 16+ CPU threads, 256GB+ of RAM. Azure SQL eDTUs: 800 to 3,000
  - Above 1TB: contact us.
Recovery model: Simple is enough and will be the fastest model. Using a Full recovery model is OK, but it might affect performance, especially during data warehouse uploads and transformations.
Compatibility level: Latest available.
Collation: the same as your source systems', if applicable. Our default is SQL_Latin1_General_CP1_CI_AS.
ETL+ requires dbo role for the MS SQL database hosting the data warehouse.
Recommended to set up SQL Encrypt connection and Trust server certificate, and its HD with encryption at rest.

Related Pages

Keywords: Data Warehouse Minimum System Requirements, Data Warehouse Specs, Data Warehouse Specifications