Lab Guide: Dify on ECS

Objective

In this lab, you will:

1. Create a Virtual Private Cloud (VPC) and configure a VSwitch
2. Create an Alibaba Cloud ECS instance
3. Deploy Dify on the ECS instance using Docker Compose
4. Build a simple Chatbot with Retrieval-Augmented Generation (RAG) application using the Qwen models.

Prerequisites

1. An Alibaba Cloud account

1      Create a VPC and VSwitch

1.1  Access the VPC Console

1. Log in to the Alibaba Cloud Console (https://home.console.alibabacloud.com/?spm=a3c0i.7911826.6791778070.31.2e6419a1K8njJA)
2. Navigate to Products and Services > Virtual Private Cloud (VPC)

1.2  Create a VPC

1. Click “VPC” on the left-side navigation pane
2. Click “Create VPC” in the top-left corner
3. Select “China (Hong Kong)” inb Region
4. Enter “{your_name}_hk_vpc” in Name
5. Enter “192.168.0.0/24” in Enter and IPv4 CIDR block

1.3  Create a VSwitch

1. Enter “{your_name}_hk_b” in Name
2. Select “Hong Kong Zone B”
3. Enter “192.168.0.32/28”
4. Click “OK”

2      Create an ECS

2.1  Create a Key Pairs

1. Go to Products and Services, select Elastic Compute Service (ECS)
2. Click “Key Pairs” on the left-side navigation pane
3. Click “Create SSH Key Pair”
4. Enter “{your_name}_key” in Name
5. Click “OK”
6. A “{your_name}_key.pem” file will be automatically downloaded in your laptop

2.2  Launch an ECS Instance

1. Go to Products and Services, select Elastic Compute Service (ECS)
2. Click Create ECS Instance
3. Select “Pay-as-you-go” in Billing Method
4. Select “China (Hong Kong)” in Region
5. Select “{your_name}_hk_vpc” & “{your_name}_hk_b” in Network adn Zone
6. Select “ecs.u1-c1m2.xlarge” in Instance
7. Click oon “Show Other Images”
8. Select “Ubuntu” & “Ubuntu 22.04 64-bit” in Image
9. Enter “40” in System Disk Size
10. Tick “Assign Public IPv4 Address” in Public IP Address
11. Select “Pay-by-traffic” in Bandwidth Billing Method
12. Select “1” in Maximum Bandwidth
13. Select “New Security Group”
14. Tick HTTP (TCP:80)
15. Select “Key Pair” in Logon Credential
16. Select “root” in Logon Username
17. Select “{your_name}_key” in Key Pair
18. Expand Advanced Setting (Optional)
19. Enter “{your_name}_hk_ecs” in Instance Name
20. Click “Create Order”
21. Click “Subscribe”

2.3  Get the ECS Public IP

1. Click “Instances” on the left-side navigation pane
2. Click on your selected instance ID
3. Look for Public IP Address under Instance Details > Configuration Information
4. Copy the Public IP Address for later usage

3      Connect to the ECS Instance

3.1  Remote Connection to ECS Instance

1. Go to Products and Services, select Elastic Compute Service (ECS)
2. Select “Instances” on the left-side navigation pane
3. Click Connect in the Actions column
4. In the Remote connection dialog box, click Sign in now in the Workbench section
5. Select “SSH Key Authentication” in the Authentication field
6. Upload the {your_name}_key.pem file downloaded in Step 2.1.16
7. Click “Log On”

3.2  Update the Server

1. Update and upgrade

apt update && apt upgrade -y

4      Install Dependencies on the Server

4.1  Install Docker

1. Install Docker

apt install -y docker.io

• Start and enable

systemctl start docker
systemctl enable docker

• Verify

docker –version

4.2  Install Docker Compose

1. Download

curl -L “https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)” -o /usr/local/bin/docker-compose

• Make executable

chmod +x /usr/local/bin/docker-compose

• Verify

docker-compose –version

4.3  Install Git

1. Install

apt install -y git

5      Deploy Dify on the Server

5.1  Clone the Dify Repository

1. Clone

git clone https://github.com/langgenius/dify.git

• Navigate

cd dify/docker

5.2  Start Dify

1. Copy the environment file

cp .env.example .env

• Launch

docker-compose up -d

• Check Status

docker ps

The services should be up as below

5.3  Access Dify Console

1. Open a browser and visit http://<PUBLIC_IP_ADDRESS> (e.g., http://47.123.45.67)
2. The Dify web interface should appear. If not, wait a minute and refresh

6      Use Case – Chatbot with RAG

6.1  Register a New Account

1. Enter your email address
2. Enter your username
3. Enter your password
4. Click “Set Up”

6.2  Login to Dify

1. Enter your username
2. Enter your password
3. Click “Sign In”

6.3  Get API Key from Model Studio

1. Log in to the Alibaba Cloud Console (https://home.console.alibabacloud.com/?spm=a3c0i.7911826.6791778070.31.2e6419a1K8njJA)
2. Navigate to Products and Services > Alibaba Cloud Model Studio
3. Click User Icon on the top right corner
4. Click “API-KEY”
5. Click “Create My API Key”
6. Select “Default Workspace”
7. Click “OK”
8. Click “View” under Actions
9. Copy the API Key under API KEY

6.4  Model Integration Settings (LLM Model)

1. Click on your username
2. Click “Settings”
3. Click “Model Provider”
4. Search “OpenAI-API-compatible”
5. Click “Install”
6. Click “Add Model”
7. Select “LLM” under Model Type
8. Enter “qwen-max” under Model Name
9. Enter “qwen-max” under Model Display Name
10. Enter the API Key copied from Step 3.3.9
11. Enter “https://dashscope-intl.aliyuncs.com/compatible-mode/v1” under API endpoint URL
12. Enter “qwen-max” under model name for API endpoint
13. Click “Save”

6.5  Model Integration Settings (Embedding Model)

1. Click “Add Model”

2. Select “Text Embedding” under Model Type

3. Enter “text-embedding-v3” under Model Name

4. Enter “text-embedding-v3” under Model Display Name

5. Enter the API Key copied from Step 3.3.9

6. Enter “https://dashscope-intl.aliyuncs.com/compatible-mode/v1” under API endpoint URL

7. Enter “text-embedding-v3” under model name for API endpoint

8. Click “Save”

6.6  Prepare a PDF Document

1. On your local computer, download the PDF file (https://www.ia.org.hk/en/legislative_framework/files/GL2_Eng.pdf)

6.7  Create a Knowledge Base

1. Go to “Knowledge” in Dify Console
2. Click “Create Knowledge”
3. Drag or upload the PDF file downloaded in Step 6.6.1
4. Click “Next”
5. Click “Preview Chunk”
6. Make sure “text-embedding-v3” is selected Embedding Model
7. Click “Save & Process”

6.8  Create a Chatbot with the Knowledge Base

1. Go to “Studio” in Dify Console
2. Click “Chatbot”
3. Click “Create from Blank”
4. Enter “{your_name}_chatbot” under App Name & Icon
5. Click “Create”
6. Click “Add” in Knowledge box
7. Add a Kownloedge box and connect between Start and Model.
8. Select the Knowledge Base created in Step 6.7
9. Select “qwen-max”
10. Use the Debug & Preview conversation box to ask questions