Managing Azure Blob Storage with Terraform

Azure Blob Storage is Microsoft’s object storage solution for the cloud. This guide demonstrates how to set up and manage Azure Blob Storage using Terraform.

Video Tutorial

Learn more about managing Azure Blob Storage with Terraform in this comprehensive video tutorial:

Prerequisites

• Azure CLI configured with appropriate permissions
• Terraform installed (version 1.0.0 or later)
• Resource group created
• Understanding of storage concepts

Project Structure

terraform-azure-storage/
├── main.tf
├── variables.tf
├── outputs.tf
├── modules/
│   └── storage/
│       ├── main.tf
│       ├── variables.tf
│       └── outputs.tf
└── policies/
    └── lifecycle.json

Storage Account Configuration

Create modules/storage/main.tf:

# Storage Account
resource "azurerm_storage_account" "main" {
  name                     = "${var.project_name}sa"
  resource_group_name      = var.resource_group_name
  location                 = var.location
  account_tier             = "Standard"
  account_replication_type = "GRS"
  account_kind            = "StorageV2"
  is_hns_enabled          = true
  
  min_tls_version         = "TLS1_2"
  enable_https_traffic_only = true

  blob_properties {
    versioning_enabled       = true
    change_feed_enabled      = true
    last_access_time_enabled = true

    container_delete_retention_policy {
      days = 7
    }

    delete_retention_policy {
      days = 30
    }
  }

  network_rules {
    default_action = "Deny"
    ip_rules       = var.allowed_ip_ranges
    virtual_network_subnet_ids = [azurerm_subnet.storage.id]
    bypass         = ["AzureServices"]
  }

  identity {
    type = "SystemAssigned"
  }

  tags = var.tags
}

# Containers
resource "azurerm_storage_container" "main" {
  for_each              = var.containers
  name                  = each.key
  storage_account_name  = azurerm_storage_account.main.name
  container_access_type = each.value.access_type
}

# Lifecycle Management Policy
resource "azurerm_storage_management_policy" "main" {
  storage_account_id = azurerm_storage_account.main.id

  rule {
    name    = "moveToArchive"
    enabled = true
    filters {
      prefix_match = ["archive/"]
      blob_types   = ["blockBlob"]
    }
    actions {
      base_blob {
        tier_to_archive_after_days = 90
        delete_after_days          = 365
      }
      snapshot {
        delete_after_days = 30
      }
      version {
        delete_after_days = 30
      }
    }
  }

  rule {
    name    = "moveToCool"
    enabled = true
    filters {
      prefix_match = ["data/"]
      blob_types   = ["blockBlob"]
    }
    actions {
      base_blob {
        tier_to_cool_after_days = 30
      }
    }
  }
}

Network Configuration

1. Virtual Network Integration

resource "azurerm_virtual_network" "main" {
  name                = "${var.project_name}-vnet"
  location            = var.location
  resource_group_name = var.resource_group_name
  address_space       = ["10.0.0.0/16"]

  subnet {
    name           = "storage-subnet"
    address_prefix = "10.0.1.0/24"
    service_endpoints = ["Microsoft.Storage"]
  }

  tags = var.tags
}

resource "azurerm_private_endpoint" "storage" {
  name                = "${var.project_name}-pe"
  location            = var.location
  resource_group_name = var.resource_group_name
  subnet_id           = azurerm_virtual_network.main.subnet.*.id[0]

  private_service_connection {
    name                           = "${var.project_name}-psc"
    private_connection_resource_id = azurerm_storage_account.main.id
    is_manual_connection          = false
    subresource_names            = ["blob"]
  }

  private_dns_zone_group {
    name                 = "default"
    private_dns_zone_ids = [azurerm_private_dns_zone.storage.id]
  }
}

resource "azurerm_private_dns_zone" "storage" {
  name                = "privatelink.blob.core.windows.net"
  resource_group_name = var.resource_group_name
}

resource "azurerm_private_dns_zone_virtual_network_link" "storage" {
  name                  = "${var.project_name}-vnet-link"
  resource_group_name   = var.resource_group_name
  private_dns_zone_name = azurerm_private_dns_zone.storage.name
  virtual_network_id    = azurerm_virtual_network.main.id
}

Security Configuration

1. Role Assignments

resource "azurerm_role_assignment" "storage_contributor" {
  scope                = azurerm_storage_account.main.id
  role_definition_name = "Storage Blob Data Contributor"
  principal_id         = var.contributor_principal_id
}

resource "azurerm_role_assignment" "storage_reader" {
  scope                = azurerm_storage_account.main.id
  role_definition_name = "Storage Blob Data Reader"
  principal_id         = var.reader_principal_id
}

2. Customer-Managed Keys

resource "azurerm_key_vault_key" "storage" {
  name         = "${var.project_name}-key"
  key_vault_id = var.key_vault_id
  key_type     = "RSA"
  key_size     = 2048

  key_opts = [
    "decrypt",
    "encrypt",
    "sign",
    "unwrapKey",
    "verify",
    "wrapKey",
  ]
}

resource "azurerm_storage_account_customer_managed_key" "main" {
  storage_account_id = azurerm_storage_account.main.id
  key_vault_id      = var.key_vault_id
  key_name          = azurerm_key_vault_key.storage.name
}

Monitoring Configuration

1. Diagnostic Settings

resource "azurerm_monitor_diagnostic_setting" "storage" {
  name                       = "${var.project_name}-diag"
  target_resource_id        = azurerm_storage_account.main.id
  log_analytics_workspace_id = var.log_analytics_workspace_id

  metric {
    category = "Transaction"
    enabled  = true

    retention_policy {
      enabled = true
      days    = 30
    }
  }

  metric {
    category = "Capacity"
    enabled  = true

    retention_policy {
      enabled = true
      days    = 30
    }
  }
}

resource "azurerm_monitor_metric_alert" "storage" {
  name                = "${var.project_name}-capacity-alert"
  resource_group_name = var.resource_group_name
  scopes              = [azurerm_storage_account.main.id]
  description         = "Alert when storage capacity exceeds threshold"

  criteria {
    metric_namespace = "Microsoft.Storage/storageAccounts"
    metric_name      = "UsedCapacity"
    aggregation      = "Average"
    operator         = "GreaterThan"
    threshold        = var.capacity_threshold_gb * 1024 * 1024 * 1024 # Convert GB to bytes
  }

  action {
    action_group_id = var.action_group_id
  }
}

Static Website Hosting

1. Static Website Configuration

resource "azurerm_storage_account" "website" {
  name                     = "${var.project_name}web"
  resource_group_name      = var.resource_group_name
  location                 = var.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
  account_kind            = "StorageV2"

  static_website {
    index_document     = "index.html"
    error_404_document = "404.html"
  }

  tags = var.tags
}

resource "azurerm_cdn_profile" "main" {
  name                = "${var.project_name}-cdn"
  location            = var.location
  resource_group_name = var.resource_group_name
  sku                 = "Standard_Microsoft"

  tags = var.tags
}

resource "azurerm_cdn_endpoint" "main" {
  name                = "${var.project_name}-cdn-endpoint"
  profile_name        = azurerm_cdn_profile.main.name
  location            = var.location
  resource_group_name = var.resource_group_name

  origin {
    name      = "primary"
    host_name = replace(replace(azurerm_storage_account.website.primary_web_endpoint, "https://", ""), "/", "")
  }

  delivery_rule {
    name  = "EnforceHTTPS"
    order = 1

    request_scheme_condition {
      operator     = "Equal"
      match_values = ["HTTP"]
    }

    url_redirect_action {
      redirect_type = "Found"
      protocol      = "Https"
    }
  }
}

Best Practices

1. Security

   • Enable firewall rules
   • Use private endpoints
   • Implement RBAC
   • Enable encryption

2. Performance

   • Choose appropriate tier
   • Enable CDN for static content
   • Configure lifecycle management
   • Monitor metrics

3. Cost Optimization

   • Use access tiers
   • Implement lifecycle policies
   • Monitor usage
   • Clean up old data

4. Data Protection

   • Enable versioning
   • Configure backup
   • Set retention policies
   • Monitor access

Advanced Features

1. Object Replication

resource "azurerm_storage_account" "destination" {
  name                     = "${var.project_name}dest"
  resource_group_name      = var.resource_group_name
  location                 = var.secondary_location
  account_tier             = "Standard"
  account_replication_type = "LRS"
  account_kind            = "StorageV2"
  is_hns_enabled          = true

  blob_properties {
    versioning_enabled = true
    change_feed_enabled = true
  }
}

resource "azurerm_storage_object_replication" "main" {
  source_storage_account_id      = azurerm_storage_account.main.id
  destination_storage_account_id = azurerm_storage_account.destination.id
  
  rules {
    source_container_name      = "source"
    destination_container_name = "destination"
    copy_blobs_created_after  = "2025-01-01T00:00:00Z"
    filter_out_blobs_with_prefix = ["temp/"]
  }
}

2. Immutable Storage

resource "azurerm_storage_container_immutability_policy" "main" {
  storage_account_id = azurerm_storage_account.main.id
  container_name     = azurerm_storage_container.main["records"].name
  
  allow_protected_append_writes = true
  period_since_creation_in_days = 365
}

Conclusion

You’ve learned how to set up and manage Azure Blob Storage using Terraform. This setup provides:

• Secure storage configuration
• Network isolation
• Monitoring and alerts
• Data lifecycle management

Remember to:

• Monitor storage metrics
• Review security settings
• Optimize costs
• Maintain compliance requirements