Managing Azure Event Grid with Terraform

Learn how to set up and manage Azure Event Grid using Terraform, including topics, subscriptions, and event handling

Managing Azure Event Grid with Terraform

Azure Event Grid is a fully managed event routing service. This guide demonstrates how to set up and manage Azure Event Grid using Terraform.

Video Tutorial

Learn more about managing Azure Event Grid with Terraform in this comprehensive video tutorial:

Prerequisites

  • Azure CLI configured with appropriate permissions
  • Terraform installed (version 1.0.0 or later)
  • Resource group created
  • Understanding of event-driven architecture

Project Structure

terraform-azure-eventgrid/
├── main.tf
├── variables.tf
├── outputs.tf
├── modules/
│   └── eventgrid/
│       ├── main.tf
│       ├── variables.tf
│       └── outputs.tf
└── policies/
    └── security.json

Event Grid Configuration

Create modules/eventgrid/main.tf:

# System Topic
resource "azurerm_eventgrid_system_topic" "storage" {
  name                   = "${var.project_name}-storage-events"
  location              = var.location
  resource_group_name   = var.resource_group_name
  source_arm_resource_id = azurerm_storage_account.main.id
  topic_type            = "Microsoft.Storage.StorageAccounts"

  identity {
    type = "SystemAssigned"
  }

  tags = var.tags
}

# Custom Topic
resource "azurerm_eventgrid_topic" "main" {
  name                = "${var.project_name}-topic"
  location            = var.location
  resource_group_name = var.resource_group_name

  input_schema = "EventGridSchema"

  public_network_access_enabled = false

  identity {
    type = "SystemAssigned"
  }

  tags = var.tags
}

# Domain
resource "azurerm_eventgrid_domain" "main" {
  name                = "${var.project_name}-domain"
  location            = var.location
  resource_group_name = var.resource_group_name

  input_schema = "EventGridSchema"

  public_network_access_enabled = false

  identity {
    type = "SystemAssigned"
  }

  tags = var.tags
}

# Domain Topic
resource "azurerm_eventgrid_domain_topic" "main" {
  name                = "orders"
  domain_name         = azurerm_eventgrid_domain.main.name
  resource_group_name = var.resource_group_name
}

# Event Subscription (to Function)
resource "azurerm_eventgrid_event_subscription" "function" {
  name                = "${var.project_name}-function-sub"
  scope               = azurerm_eventgrid_topic.main.id
  included_event_types = ["OrderCreated", "OrderUpdated"]

  azure_function_endpoint {
    function_id = "${var.function_app_id}/functions/ProcessOrder"
    max_events_per_batch = 1
    preferred_batch_size_in_kilobytes = 64
  }

  advanced_filter {
    string_begins_with {
      key    = "Subject"
      values = ["orders/high-priority"]
    }
  }

  retry_policy {
    max_delivery_attempts = 30
    event_time_to_live   = 1440
  }

  dead_letter_identity {
    type = "SystemAssigned"
  }

  storage_blob_dead_letter_destination {
    storage_account_id          = azurerm_storage_account.dlq.id
    storage_container_name      = "deadletter"
    storage_blob_container_sas_url = data.azurerm_storage_account_blob_container_sas.dlq.sas
  }
}

# Event Subscription (to Service Bus)
resource "azurerm_eventgrid_event_subscription" "servicebus" {
  name                = "${var.project_name}-servicebus-sub"
  scope               = azurerm_eventgrid_topic.main.id
  included_event_types = ["OrderCancelled"]

  service_bus_topic_endpoint_id = azurerm_servicebus_topic.main.id

  advanced_filter {
    string_in {
      key    = "Data.reason"
      values = ["OutOfStock", "CustomerRequest"]
    }
  }

  delivery_property {
    header_name = "OrderId"
    type        = "Dynamic"
    source_field = "data.orderId"
  }
}

# Event Subscription (to Event Hub)
resource "azurerm_eventgrid_event_subscription" "eventhub" {
  name                = "${var.project_name}-eventhub-sub"
  scope               = azurerm_eventgrid_topic.main.id
  included_event_types = ["OrderShipped"]

  eventhub_endpoint_id = azurerm_eventhub.main.id

  advanced_filter {
    number_greater_than {
      key    = "Data.amount"
      value  = 1000
    }
  }
}

# Event Subscription (to Storage Queue)
resource "azurerm_eventgrid_event_subscription" "queue" {
  name                = "${var.project_name}-queue-sub"
  scope               = azurerm_eventgrid_topic.main.id
  included_event_types = ["OrderReturned"]

  storage_queue_endpoint {
    storage_account_id = azurerm_storage_account.main.id
    queue_name        = "returns"
  }
}

Network Configuration

  1. Virtual Network Integration
resource "azurerm_virtual_network" "main" {
  name                = "${var.project_name}-vnet"
  location            = var.location
  resource_group_name = var.resource_group_name
  address_space       = ["10.0.0.0/16"]

  subnet {
    name           = "eventgrid-subnet"
    address_prefix = "10.0.1.0/24"
    service_endpoints = ["Microsoft.EventGrid"]
  }

  tags = var.tags
}

resource "azurerm_private_endpoint" "eventgrid" {
  name                = "${var.project_name}-pe"
  location            = var.location
  resource_group_name = var.resource_group_name
  subnet_id           = azurerm_virtual_network.main.subnet.*.id[0]

  private_service_connection {
    name                           = "${var.project_name}-psc"
    private_connection_resource_id = azurerm_eventgrid_topic.main.id
    is_manual_connection          = false
    subresource_names            = ["topic"]
  }

  private_dns_zone_group {
    name                 = "default"
    private_dns_zone_ids = [azurerm_private_dns_zone.eventgrid.id]
  }
}

resource "azurerm_private_dns_zone" "eventgrid" {
  name                = "privatelink.eventgrid.azure.net"
  resource_group_name = var.resource_group_name
}

resource "azurerm_private_dns_zone_virtual_network_link" "eventgrid" {
  name                  = "${var.project_name}-vnet-link"
  resource_group_name   = var.resource_group_name
  private_dns_zone_name = azurerm_private_dns_zone.eventgrid.name
  virtual_network_id    = azurerm_virtual_network.main.id
}

Security Configuration

  1. Role Assignments
resource "azurerm_role_assignment" "eventgrid_publisher" {
  scope                = azurerm_eventgrid_topic.main.id
  role_definition_name = "EventGrid Data Sender"
  principal_id         = var.publisher_principal_id
}

resource "azurerm_role_assignment" "eventgrid_subscriber" {
  scope                = azurerm_eventgrid_topic.main.id
  role_definition_name = "EventGrid Data Receiver"
  principal_id         = var.subscriber_principal_id
}
  1. IP Rules
resource "azurerm_eventgrid_topic_inbound_ip_rule" "main" {
  eventgrid_topic_name = azurerm_eventgrid_topic.main.name
  resource_group_name  = var.resource_group_name
  ip_mask             = var.allowed_ip_range
  action              = "Allow"
}

Monitoring Configuration

  1. Diagnostic Settings
resource "azurerm_monitor_diagnostic_setting" "eventgrid" {
  name                       = "${var.project_name}-diag"
  target_resource_id        = azurerm_eventgrid_topic.main.id
  log_analytics_workspace_id = var.log_analytics_workspace_id

  log {
    category = "DeliveryFailures"
    enabled  = true

    retention_policy {
      enabled = true
      days    = 30
    }
  }

  metric {
    category = "AllMetrics"
    enabled  = true

    retention_policy {
      enabled = true
      days    = 30
    }
  }
}

resource "azurerm_monitor_metric_alert" "eventgrid" {
  name                = "${var.project_name}-delivery-alert"
  resource_group_name = var.resource_group_name
  scopes              = [azurerm_eventgrid_topic.main.id]
  description         = "Alert when event delivery fails"

  criteria {
    metric_namespace = "Microsoft.EventGrid/topics"
    metric_name      = "UnmatchedEventCount"
    aggregation      = "Total"
    operator         = "GreaterThan"
    threshold        = 10
  }

  action {
    action_group_id = var.action_group_id
  }
}

Advanced Features

  1. Input Mapping
resource "azurerm_eventgrid_event_subscription" "mapping" {
  name                = "${var.project_name}-mapping-sub"
  scope               = azurerm_eventgrid_topic.main.id

  input_mapping_fields {
    topic      = "subject"
    event_type = "eventType"
    event_time = "eventTime"
    data_version = "dataVersion"
  }

  input_mapping_default_values {
    data_version = "1.0"
    subject     = "DefaultSubject"
  }
}
  1. Batching
resource "azurerm_eventgrid_event_subscription" "batch" {
  name                = "${var.project_name}-batch-sub"
  scope               = azurerm_eventgrid_topic.main.id

  webhook_endpoint {
    url = "https://example.com/api/events"
    max_events_per_batch = 10
    preferred_batch_size_in_kilobytes = 64
  }
}

Best Practices

  1. Performance

    • Use appropriate endpoints
    • Configure batching
    • Implement retry policies
    • Monitor delivery rates

  2. Security

    • Enable private endpoints
    • Use RBAC
    • Implement IP rules
    • Enable authentication

  3. Reliability

    • Configure dead-lettering
    • Set retry policies
    • Monitor failures
    • Use filtering

  4. Cost Optimization

    • Clean up subscriptions
    • Use appropriate endpoints

Conclusion

You’ve learned how to set up and manage Azure Event Grid using Terraform. This setup provides:

  • Event routing
  • Security and filtering
  • Monitoring and alerts
  • Dead-letter handling

Remember to:

  • Monitor delivery rates
  • Review security settings
  • Handle dead-letters
  • Update access controls