Module: Implement AI Solutions with Foundry

Section: Azure AI Foundry Portal

Lesson Title: Creating Projects and Workspaces

Introduction: The Foundation of AI Development

In the modern landscape of software engineering, moving from a local prototype to a production-ready artificial intelligence application requires more than just high-quality code. It requires an environment that manages infrastructure, security, data privacy, and model lifecycle management. Azure AI Foundry serves as the unified platform designed to bridge this gap, acting as the central hub where developers orchestrate their machine learning and generative AI workflows.

Understanding how to properly structure your environment within Azure AI Foundry is the most critical first step in your journey toward building reliable AI solutions. When we talk about "Projects" and "Workspaces," we are talking about the architectural bedrock of your cloud-based AI operations. A workspace acts as your primary resource container, while a project serves as the focused sandbox where you collaborate on specific tasks, experiment with prompts, and refine model deployments.

Why does this matter? If you fail to organize your workspace correctly from the start, you will quickly face issues with resource fragmentation, cost tracking, and security governance. By mastering the setup process, you ensure that your team can iterate quickly without stepping over each other’s work, maintain clear separation between testing and production environments, and keep a tight handle on your cloud expenditure. This lesson will guide you through the technical intricacies of creating and configuring these resources, ensuring your path to deployment is as clear as possible.

Understanding the Hierarchy: Resource Groups, Workspaces, and Projects

Before we dive into the interface, it is essential to visualize the hierarchy of Azure resources. Azure AI Foundry does not exist in a vacuum; it sits atop the standard Azure Resource Manager (ARM) model. Understanding this hierarchy prevents the common mistake of over-provisioning or creating security silos that are difficult to manage later.

1. Azure Subscription: The top-level billing container. All costs associated with your AI development will roll up to this subscription.
2. Resource Group: A logical container that holds related resources for an Azure solution. We typically group the AI workspace, storage accounts, and key vaults here.
3. Azure AI Workspace: The top-level resource for AI development. It manages the storage, compute, and networking configurations required for your projects.
4. AI Project: A child resource of the workspace. This is where you conduct your day-to-day work, such as prompt engineering, fine-tuning, and model evaluation.

Callout: Workspace vs. Project - What is the difference? Think of the Workspace as your office building. It provides the security, the electricity (compute), and the filing cabinets (storage) for your team. The Project is your specific desk or conference room within that building. You can have many projects inside one workspace, all sharing the same security policies and compute resources, which keeps your management overhead low while maintaining logical separation for different AI tasks.

Step-by-Step: Provisioning Your First Azure AI Workspace

Provisioning a workspace is the gateway to using Azure AI Foundry. While you can do this via the Azure CLI or Terraform, we will start with the Azure AI Foundry Portal interface to ensure you understand the underlying parameters that influence your workspace’s behavior.

1. Preparation

Before clicking "Create," ensure you have the necessary permissions. You must be an Owner or Contributor at the Resource Group level to create an Azure AI resource. Additionally, ensure your subscription is registered for the Microsoft.MachineLearningServices provider.

2. Configuration Steps

Navigate to the Azure AI Foundry portal and select "Create New Project." However, if a workspace does not exist, you will be prompted to create the workspace first.

• Subscription: Select the billing subscription you wish to associate with the project.
• Resource Group: Create a new one or select an existing group. Using a dedicated resource group for AI projects is a best practice for clean lifecycle management.
• Region: This is critical. Choose a region that supports the specific models you intend to use. Not every region has the same GPU availability or model support (e.g., GPT-4o availability).
• Storage and Key Vault: Azure will automatically suggest creating a new Azure Storage Account and Azure Key Vault. Accept these defaults unless your enterprise security policy dictates the use of existing, pre-configured infrastructure.

Tip: Choosing a Region Always check the "Azure AI services region availability" documentation before provisioning. If you are building an application that requires low latency, ensure your AI workspace is in the same region as your application servers. If you are subject to data residency requirements, ensure the region aligns with your local compliance laws.

Configuring the AI Project: Your Development Sandbox

Once the workspace is initialized, the project is your primary interface. Within the project, you define the "what" and the "how" of your AI solution. A project is where you track your prompt flow, manage your model catalog, and monitor evaluation metrics.

Key Settings in Project Creation

When you define a new project in the Foundry portal, you are essentially setting up the metadata for your work. You will be asked to name the project and provide a description. While this sounds trivial, maintain a clear naming convention (e.g., proj-customer-support-bot vs. test-experiment-01).

After creating the project, you are presented with the Project Dashboard. This dashboard is your command center. You will see tabs for:

• Prompt Flow: Where you build and test logic chains.
• Models: The catalog where you select base models to deploy.
• Evaluation: Where you run tests against your model outputs to ensure quality.
• Deployments: Where you manage the live endpoints for your models.

Infrastructure as Code (IaC) for AI Projects

Relying solely on the portal for resource creation is fine for learning, but it is not suitable for professional environments. Using Infrastructure as Code (IaC) ensures that your workspaces are reproducible, version-controlled, and consistent across development, staging, and production.

Example: Provisioning via Azure CLI

The Azure CLI is the most straightforward way to automate the creation of your workspace and project. Below is a script to create a workspace and a project in one go.

# Define your variables
RESOURCE_GROUP="rg-ai-development"
LOCATION="eastus"
WORKSPACE_NAME="ws-innovation-hub"
PROJECT_NAME="proj-data-analyzer"

# Create the Resource Group
az group create --name $RESOURCE_GROUP --location $LOCATION

# Create the Azure AI Workspace
az ml workspace create \
    --name $WORKSPACE_NAME \
    --resource-group $RESOURCE_GROUP \
    --location $LOCATION

# Create the AI Project within that Workspace
az ml project create \
    --name $PROJECT_NAME \
    --workspace-name $WORKSPACE_NAME \
    --resource-group $RESOURCE_GROUP

Understanding the Code:

• az group create: This establishes the container for all your resources.
• az ml workspace create: This initializes the core AI service. It sets up the underlying storage and key vault references.
• az ml project create: This attaches a logical project to the workspace, enabling you to isolate your experiments from other projects in the same workspace.

Best Practices for Workspace and Project Management

To keep your Azure AI Foundry environment healthy, you must adhere to several industry-standard practices. These practices prevent "resource sprawl" and ensure your costs remain predictable.

1. Implement Tagging

Always apply tags to your resource groups and workspaces. Tags like Environment: Production, Owner: DataScienceTeam, and ProjectCode: XYZ allow you to track costs and manage access control efficiently.

2. Networking and Security

By default, AI workspaces are accessible via the public internet. For enterprise applications, this is often insufficient.

• Private Endpoints: Configure your workspace to use Azure Private Link. This ensures that traffic between your virtual network and the AI service never leaves the Microsoft backbone network.
• Managed Identities: Avoid using API keys whenever possible. Use Managed Identities to grant your application access to the AI workspace. This eliminates the need to manage secret rotation manually.

3. Lifecycle Management of Compute

Compute resources are the most expensive part of your AI setup. If you use integrated compute (like managed instances), ensure you set up auto-shutdown policies.

Warning: Cost Management It is very easy to forget about an active model deployment. Always check the "Deployments" tab in your project to ensure you aren't paying for high-throughput endpoints that aren't being used. Implement alerts in Azure Cost Management to notify you when spending exceeds a specific threshold.

Comparison: Manual vs. Automated Provisioning

Common Pitfalls and How to Avoid Them

Even experienced engineers run into issues when setting up Azure AI Foundry for the first time. Here are the most frequent mistakes and how to navigate them.

Pitfall 1: Ignoring Quota Limits

Many developers try to deploy a massive model (like GPT-4o) only to find they don't have the "quota" in their subscription. Azure limits the number of concurrent requests and GPU capacity by default to prevent runaway costs.

• Solution: Check your "Usage + Quotas" section in the Azure portal before you start. If you need more, you can request a quota increase through the portal. Do this early, as it can take up to 48 hours for a request to be processed.

Pitfall 2: Over-provisioning Projects

Some teams create a new project for every single experiment. This leads to a messy portal interface and makes it difficult to track where your best prompts are located.

• Solution: Follow a "logical grouping" strategy. Create one project for "Customer Service Chatbot" and keep all iterations, prompts, and evaluation runs for that specific bot within that single project.

Pitfall 3: Hardcoding Credentials

It is tempting to put your connection strings directly into your code. This is a massive security risk.

• Solution: Always use Azure Key Vault to store your connection strings and use the DefaultAzureCredential class from the Azure SDK to authenticate your application.

Callout: The "Principle of Least Privilege" When assigning access to your AI workspace, do not simply give everyone "Contributor" access. Use Role-Based Access Control (RBAC). A data scientist might need "Azure AI Developer" access to create prompts, but they might not need the "Owner" role which allows them to delete the entire workspace. Always limit permissions to only what is required for the user's specific tasks.

Deep Dive: Monitoring and Operations

Once your project is running, monitoring the health of your deployments becomes a full-time task. Azure AI Foundry integrates directly with Azure Monitor and Log Analytics, providing deep visibility into the performance of your AI models.

Setting up Log Analytics

When you create your workspace, you have the option to link it to a Log Analytics Workspace. Do not skip this step. Without it, you cannot monitor the latency of your model calls, the token usage, or the frequency of errors.

Once linked, you can write Kusto Query Language (KQL) queries to analyze your AI traffic:

// Example: Find the average latency of model calls
AmlOnlineEndpointRequests
| summarize avg(DurationMs) by ModelName, bin(TimeGenerated, 1h)
| render timechart

This level of detail is necessary for debugging production issues. If a user reports that the chatbot is "slow," you can use this data to determine if the delay is in the model response time or the network overhead.

Managing Model Catalog and Deployments

A core feature of the Azure AI Foundry portal is the Model Catalog. This is where you select the models you want to deploy. Whether you are using OpenAI models, Llama, or Mistral, the process is consistent.

Step-by-Step: Deploying a Model

1. Navigate to the Model Catalog: Within your project, click on the "Models" tab.
2. Select your model: Browse or search for the model version you need (e.g., gpt-4o-2024-05-13).
3. Click "Deploy": You will be presented with deployment options.
   • Serverless API: This is the easiest path. You pay for what you use, and Microsoft manages the underlying hardware.
   • Managed Compute: You provision a specific cluster of VMs. This is better for high-throughput, predictable workloads where you want dedicated resources.
4. Configure Instance Type: Choose the VM size. For small experiments, start with a smaller size to save costs. You can always scale up later.

Note: Always check for "Model versioning." When you deploy a model, you are often deploying a specific version. If you want to take advantage of new features or security patches, you will need to update your deployment configuration.

Best Practices for Collaboration

Azure AI Foundry is designed for teams. When multiple people are working in the same project, coordination is key.

• Version Control for Prompts: Use the "Prompt Flow" feature to save your prompts as versioned files. You can check these files into a Git repository just like your source code.
• Shared Evaluation Datasets: Create a central folder in your project for evaluation datasets. This ensures that every team member is testing their model iterations against the same benchmarks.
• Documentation: Use the "Description" fields in your project and deployments. Write down why a specific model was chosen or why a specific configuration was applied. This saves hours of effort for team members who join the project later.

Troubleshooting Common Setup Issues

If you find that your project isn't working as expected, follow this checklist:

1. Check Permissions: If you cannot see the "Deploy" button, you likely lack the Azure AI Developer or Contributor role.
2. Verify Network Connectivity: If your application cannot connect to the workspace, check if your local machine's IP is allowed in the firewall settings, or if you are behind a corporate VPN that blocks the Azure endpoints.
3. Validate Quota: If you get a "429 Too Many Requests" or a generic "Deployment Failed" error, check your subscription quota.
4. Review Resource Provider Registration: Occasionally, a new subscription is not fully initialized for AI services. Run az provider register --namespace Microsoft.MachineLearningServices to ensure the provider is active.

Summary and Key Takeaways

Creating and managing Azure AI workspaces and projects is a foundational skill for any AI engineer. By moving beyond the "click-through" approach and embracing structured, automated, and secure configurations, you set your projects up for long-term success.

Key Takeaways for Success:

• Hierarchy Matters: Understand the relationship between Subscriptions, Resource Groups, Workspaces, and Projects. Proper organization from day one prevents a chaotic cloud environment.
• Automation is Non-Negotiable: Use the Azure CLI or Terraform to define your infrastructure. Manual portal setup is for exploration; code is for production.
• Security First: Use Managed Identities and Private Endpoints to secure your AI assets. Never store API keys in your source code.
• Monitor Everything: Always link your workspace to a Log Analytics workspace. Without telemetry, you are flying blind in production.
• Cost Awareness: Treat AI compute as a finite resource. Use tags, alerts, and auto-shutdown policies to keep your monthly bill under control.
• Version Control Your Logic: Treat your prompts and evaluation configurations as code. Use Git to track changes in your Prompt Flow files.
• Start Small, Scale Up: Begin with serverless deployments to test functionality before committing to dedicated, high-cost compute clusters.

By following these principles, you will be able to navigate the Azure AI Foundry portal with confidence, ensuring that your AI solutions are not only powerful but also secure, scalable, and manageable. Remember that the platform is constantly evolving; keep an eye on the official documentation for updates to model support and new features that can further streamline your workflow.