LLM Chatbot Playground

This blueprint generates a sample chatbot using the Anthropic company's LLM Claude 2, one of the foundational models provided by Amazon Bedrock for generative AI.

Basic Conversation

Bot Personalization

Add your own instruction and give external knowledge as URL or files (a.k.a RAG). The bot can be shared among application users.

📚 Supported Languages

• English 💬
• 日本語 💬 (ドキュメントはこちら)
• 한국어 💬
• 中文 💬

🚀 Super-easy Deployment

• On us-east-1, open Bedrock Model access > Manage model access > Check Anthropic / Claude, Anthropic / Claude Instant and Cohere / Embed Multilingual then Save changes.

Architecture

It's an architecture built on AWS managed services, eliminating the need for infrastructure management. Utilizing Amazon Bedrock, there's no need to communicate with APIs outside of AWS. This enables deploying scalable, reliable, and secure applications.

• Amazon DynamoDB: NoSQL database for conversation history storage
• Amazon API Gateway + AWS Lambda: Backend API endpoint (AWS Lambda Web Adapter, FastAPI)
• Amazon SNS: Used to decouple streaming calls between API Gateway and Bedrock because streaming responses can take over 30 seconds in total, exceeding the limitations of HTTP integration (See quota).
• Amazon CloudFront + S3: Frontend application delivery (React, Tailwind CSS)
• AWS WAF: IP address restriction
• Amazon Cognito: User authentication
• Amazon Bedrock: Managed service to utilize foundational models via APIs. Claude is used for chat response and Cohere for vector embedding
• Amazon EventBridge Pipes: Receiving event from DynamoDB stream and launching ECS task to embed external knowledge
• Amazon Elastic Container Service: Run crawling, parsing and embedding tasks. Cohere Multilingual is the model used for embedding.
• Amazon Aurora PostgreSQL: Scalable vector store with pgvector plugin

Connections and permissions

This blueprint supports the Amazon CodeCatalyst Development Role, which can be created from the AWS console Codecatalyst management application. When clicking "add IAM role", the first option is to create a CodeCatalyst development role. This role can be used across multiple blueprints.

The other option is creating a blueprint specific IAM role, which can be added to the Amazon CodeCatalyst space by selecting "Add an existing IAM role" from the add IAM role options. The IAM role needs to contain the CodeCatalyst trust policy, as well as the following permissions:

{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Action": [
            "iam:DeleteRole",
            "iam:GetRole",
            "iam:TagRole",
            "iam:CreateRole",
            "iam:AttachRolePolicy",
            "iam:DetachRolePolicy",
            "iam:DeleteRolePolicy",
            "cloudformation:*",
            "lambda:*",
            "apigateway:*",
            "ecr:*",
            "ssm:PutParameter",
            "ssm:DeleteParameter",
            "iam:PutRolePolicy",
            "s3:*",
            "ssm:GetParameter",
            "ssm:GetParameters"
          ],
          "Resource": "*"
      },
      {
          "Effect": "Allow",
          "Action": [
              "sts:AssumeRole"
          ],
          "Resource": [
              "arn:aws:iam::*:role/cdk-*"
          ]
      }
  ]
}

The IAM roles also require the Amazon CodeCatalyst service principals codecatalyst.amazonaws.com and codecatalyst-runner.amazonaws.com.

Required IAM role trust policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "codecatalyst.amazonaws.com",
                    "codecatalyst-runner.amazonaws.com"
                ]
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Features and Roadmap

Basic chat features

• Authentication (Sign-up, Sign-in)
• Creation, storage, and deletion of conversations
• Copying of chatbot replies
• Automatic subject suggestion for conversations
• Syntax highlighting for code
• Rendering of Markdown
• Streaming Response
• IP address restriction
• Edit message & re-send
• I18n
• Model switch (Claude Instant / Claude)

Customized bot features

• Customized bot creation
• Customized bot sharing

RAG features

• Web (html)
• Text data (txt, csv, markdown and etc)
• PDF
• Microsoft office files (pptx, docx, xlsx)
• Youtube transcript

Admin features

• Admin console to analyze user usage

RAG (Retrieval Augmented Generation)

See here.

Authors

• Takehiro Suzuki
• Yusuke Wada

License

This library is licensed under the MIT-0 License. See the LICENSE file.

When you create or update a project using a blueprint, CodeCatalyst may generate resources such as source repository, sample source code, CI/CD workflows, build and test reports, secrets, integrated issue tracking tools, etc. You should review the generated artifacts/project before deploying to a production or publicly accessible environment. You are responsible for all activities in your production and publicly accessible environments.