High Level Architecture and Design

The framework architecture is built upon the following building blocks:

• AI powered application
  
  • Python code
  • OpenAI Python SDK
• Docker based deployment LLM Guard API
  
  • Docker base deployment
  • AWS EC2 c5a.8xlarge instance as a hosting VM
• Azure OpenAI service
  
  • Large language model gpt-35-turbo deployment created on Azure OpenAI Studio

Infrastructure Setup

The following is the summary of the infrastructure setup:

• The Azure OpenAI service is deployed and its access from Internet is controlled using the following:
  
  • Keys and endpoint
  • Networking firewall rules to allow access from selected IP ranges
• LLM Guard API is deployed using the laiyer/llm-guard-api Docker image. Its Docker container is hosted on AWS EC2 c5a.8xlarge instance. The API access is made accessible from the Internet via an AWS Elastic IP address associated with the EC2 instance and restricted using the API AUTH_TOKEN.

Implementation

• LLM Guard API Deployment on EC2 Ubuntu
• Direct Azure OpenAI Model Access Script
• LLM Guard API Usage Script
• AI Powered Application and LLM Guard Integration Script