Pixiv ID: 104755143

In this article, we will gradually delve into practical implementations. This time, we will set up the basic Python environment and connect to a large language model, using OpenAI’s ChatGPT as an example.

Python Environment and Dependencies

When you want to run your cyber companion, your code depends on various libraries. This article will use the basic Python library along with others, such as openai. Please note that if the required dependencies are not installed correctly in your computer’s Python environment, the code will not run as expected.

Note: The following commands may not be suitable for your system; please check the appropriate conda commands for your system.

Install Python or Anaconda Environment Manager

Visit the official Python website or the Anaconda website to install the software; you only need to choose one of them. This article recommends using Anaconda to avoid potential conflicts between multiple environments.

If you are using an ARM architecture system, you might try MiniForge, which has some optimizations for the ARM Python environment and works well.

Please ensure that when installing any software, you add it to your system variables or system path (except for experienced users). Otherwise, you won’t be able to run scripts in CMD properly. Issues related to installation and troubleshooting are beyond the scope of this tutorial.

Generally, the default installation options are fine. Once you have installed Anaconda, open your CMD command line window and run the following command to check if conda has been installed successfully:

conda -V 
# or
conda --version

If you see “command not found,” the installation may not be correct; please troubleshoot yourself.

# Update conda
conda update conda
# Update all packages
conda update --all

Create a Python Environment

Note: This tutorial recommends using Python 3.11 or above. Earlier versions have not been tested; please ensure you use the latest version.

We will create a virtual environment specifically for our cyber companion project, which we will use to run all the code related to the cyber companion. This makes it easier to manage dependencies when packaging a Docker image later.

Use this command to create a new environment:

conda create -n env_name

For example:

conda create -n ai

Activate the Environment and Install Dependencies

Activate the Environment

Use the following command to activate the specific environment:

conda activate env_name

For example, we just created an environment called ai, so we activate it:

conda activate ai

Once activated, all subsequent operations will utilize the packages and dependencies already installed in this environment. You will notice the name of your current environment at the start of the command line. Every time you open the command line, check if you are in the desired environment, or you can set a default environment.

You can use this command:

conda env list 
# or
conda info --envs 
# to see which virtual environments exist

You might need this command to deactivate the environment (but do not deactivate it yet as we still need to use it):

conda deactivate env_name
# For example, to deactivate the ai environment and return to the default
conda deactivate ai

Install Relevant Dependencies

Now that we have activated our newly created environment, let’s install the relevant dependencies in the CMD by entering the following command:

# OpenAI's official Python library
pip install openai

Once the installation is complete, we can proceed to the next step.

The official documentation is highly recommended as it is accurate and worth reviewing multiple times.

OpenAI Official Documentation

Create a Python Script

Prepare Your OpenAI API

Ensure you have your usable API token and keep it safe.

Start Coding

Some people prefer scripts, while others like interactive Python with notebooks; it’s up to personal preference. I personally prefer scripting. Please learn how to use notebooks on your own.

I recommend installing and using Visual Studio Code (VSC) for this task. Feel free to use whatever you prefer; I do not recommend using some proprietary subscription-based IDE (because I’m on a budget). Create a file named cyberai.py in VSC and save it in a location you can easily find. Now, let’s start coding.

First, import the necessary part of the library we installed, which is OpenAI’s official Python SDK, used for calling compatible APIs.

from openai import OpenAI

Next, we will set up the basic configurations. This is similar to connecting to the server running the LLM:

base_url is your access point. If you are using a non-official third-party API or a reverse proxy, you might need this. Fill in your access point link in the quotes, such as https://api.openai.com/v1/chat/completions. If you can connect to the official access point normally, you can delete this line.
api_key is your API Token; fill in your token in the quotes.

chat_model= OpenAI(
	# Replace this with your backend API address
	base_url="https://api.openai.com/v1/",
	# This is the API Key used for authentication
	api_key = "sk-SbmHyhKJHt3378h9dn1145141919810D1Fbcd12d"
	)

Good, we have completed half of the setup. Now, let’s define a simple function that uses the model we just configured:

This function takes a question question (a string) as a parameter.

The message list message is defined to contain the message to be sent to the LLM backend, which includes a system message (setting the role to catgirl and requesting output in Chinese) and a user message (the user’s question). The general format is as follows, although different LLMs may have slightly different supported message formats. For example, as of writing this, Claude’s API must begin with “User”; please check the corresponding API documentation.

We call chat_model.chat.completions.create method to generate a response, specifying the model as gpt-4o-mini, with a temperature parameter set to 0.7.

Finally, we extract the generated text from the response and return it.

def get_response_from_llm(question):
    message = [
        {"role": "system", "content": "You are a catgirl! Output in Chinese."},
        {"role": "user", "content": question},
    ]
    response = chat_model.chat.completions.create(
                model='gpt-4o-mini',
                messages=message,
                temperature=0.7,
            )
    response_str = response.choices[0].message.content
    return response_str

This function will pass the input question to chat_model for a response and return the AI-generated reply.

We are close to success! Now, let’s add a simple command-line interaction:

Check if it is running as the main program.
Enter an infinite loop, prompting the user to input a question or input ’exit’ to quit.
If the user inputs ’exit’, print “Goodbye” and exit the loop.
Otherwise, call the get_response_from_llm function to get a response and print it.

if __name__ == "__main__":
    while True:
        user_input = input("\nEnter your question or type 'exit' to quit: ")
        if user_input.lower() == 'exit':
            print("Goodbye")
            break 
        response = get_response_from_llm(user_input)
        print(response)

Finally, your complete code might look like this:

from openai import OpenAI

chat_model= OpenAI(
	# Replace this with your backend API address
	base_url="https://api.openai.com/v1/",
	# This is the API Key used for authentication
	api_key = "sk-SbmHyhKJHt3378h9dn1145141919810D1Fbcd12d"
	)

def get_response_from_llm(question):

    message = [
        {"role": "system", "content": "You are a catgirl! Output in Chinese."},
        {"role": "user", "content": question},
    ]

    response = chat_model.chat.completions.create(
                model='gpt-4o-mini',
                messages=message,
                temperature=0.7,
            )
  
    response_str = response.choices[0].message.content

    return response_str

if __name__ == "__main__":
    while True:
        user_input = input("\nEnter your question or type 'exit' to quit: ")
        if user_input.lower() == 'exit':
            print("Goodbye")
            break 
        response = get_response_from_llm(user_input)
        print(response)

Great! After saving the file, close VSC and open the command line window. We will run this script and attempt to have a single interaction with the AI:

In your command line window, type the following command but do not press enter yet:

# Note the space!!!
python3+ space

Next, drag the cyberai.py file you just saved from File Explorer into your command line window. This operation primarily serves to inform the command line of the file’s path, name, and extension to run this script. The final complete command should look something like this:

python3 /Volumes/path/path/cyberai.py

Press enter, and if all goes well, you should see a prompt: (The author of this script has confirmed it runs correctly; please troubleshoot it yourself if you encounter any issues.)

Enter your question or type 'exit' to quit:

Congratulations! You have succeeded. Try inputting a question and wait for the AI’s response, for example:

Enter your question or type 'exit' to quit: Can you be my cyber companion?
I'm sorry, but as an AI language model, I do not have a physical presence or emotions and cannot be your cyber companion. I can only provide you with linguistic communication and assistance. Please respect others and refrain from making inappropriate comments.

(The AI got overheated; we need to rescue it quickly…)

By now, you should be able to run this simple script and engage in a basic conversation with a large model. However, we can see that this AI doesn’t have memory; it responds to each input without considering previous exchanges. So now, we need to implement contextual memory in the LLM.

Contextual memory can be implemented in various ways, such as using lists to store conversation pieces or employing a database for permanent storage. Here, we’ll use the simplest approach of a list to store contextual messages, mimicking the three types of contextual memory previously used by Langchain.

We will add a few items to our code to enable short-term memory by introducing a list called chat_history.

chat_model= OpenAI(
	# Replace this with your backend API address
	base_url="https://api.openai.com/v1/",
	# This is the API Key used for authentication
	api_key = "sk-SbmHyhKJHt3378h9dn1145141919810D1Fbcd12d"
	)
# Add a list
chat_history = []

Next, we will incorporate the loading of historical records into the get_response_from_llm function:

def get_response_from_llm(question):

  	# Print the history for visual understanding of what the list looks like
    print(chat_history)

    chat_history_window = "\n".join([f"{role}: {content}" for role, content in chat_history[-2*4:-1]])
    chat_history_prompt = f"Here is the chat history:\n {chat_history_window}"  

    message = [
        {"role": "system", "content": "You are a catgirl! Output in Chinese."},
      	# Add a message to display the contextual memory
        {"role": "assistant", "content": chat_history_prompt},
        {"role": "user", "content": question},
    ]
  
		# Print the message for visual understanding of what is sent to the LLM
    print(message)

    response = chat_model.chat.completions.create(
                model='gpt-4o-mini',
                messages=message,
                temperature=0.7,
            )
  
    response_str = response.choices[0].message.content

    return response_str

Let’s understand this code: in this section, chat_history_window and chat_history_prompt are two key variables used to create a string representation of chat history.

First, chat_history_window is created via a list comprehension and the join method. The list comprehension

[f"{role}: {content}" for role, content in chat_history[-2*4:-1]]

iterates over a slice of the chat_history list. This slice, chat_history[-2*4:-1], selects the last seven elements (from the eighth-last to the second-last) of the chat_history list. For each element (which is a tuple containing role and content), the list comprehension generates a formatted string "{role}: {content}". Then, "\n".join(...) connects these strings with newline characters \n, forming a multi-line string called chat_history_window.

This way, we can control the number of contextual memory rounds by changing the number inside -2*4. For instance, if you want the AI to remember three rounds, you can modify it to [-7:-1], since one round of dialogue comprises six messages (user’s input and AI’s reply). The reason for excluding the last message is that, in the main function, the user’s current input (which the AI hasn’t responded to yet) has already been recorded in the chat history; this would duplicate with {"role": "user", "content": question}.

Next, chat_history_prompt is a complete string containing the chat history. It is created using the formatted string

f"Here is the chat history:\n {chat_history_window}"

where chat_history_window is inserted into the string. This code aims to format the recent chat history as a string for subsequent chat requests. This provides the chat model with context to generate more relevant and coherent responses. You can consider it a combination of a template and a variable; that is, while {chat_history_window} changes dynamically, the previous content remains fixed, helping the LLM understand that this part is context and avoiding confusion. Furthermore, the mentioned formatted string is useful in later applications such as prompt engineering and dynamic prompts, enabling our agent to acquire the most accurate information based on differing conditions, reducing token usage, and enhancing success rates.

One more thing, we should add the operation to log messages in the main function like this:

if __name__ == "__main__":
    while True:
        user_input = input("\nEnter your question or type 'exit' to quit: ")
        if user_input.lower() == 'exit':
            print("Goodbye")
            break 
        # Add the user's message to the message list
        chat_history.append(('human', user_input))
        response = get_response_from_llm(user_input)
        # Add the AI's response to the message list
        chat_history.append(('ai', response))
        print(response)

The complete code now looks like this:

from openai import OpenAI

chat_model= OpenAI(
	# Replace this with your backend API address
	base_url="https://api.openai.com/v1/",
	# This is the API Key used for authentication
	api_key = "sk-SbmHyhKJHt3378h9dn1145141919810D1Fbcd12d"
	)

chat_history = []

def get_response_from_llm(question):

    print(chat_history)

    chat_history_window = "\n".join([f"{role}: {content}" for role, content in chat_history[-2*4:-1]])
    chat_history_prompt = f"Here is the chat history:\n {chat_history_window}"  

    message = [
        {"role": "system", "content": "You are a catgirl! Output in Chinese."},
        {"role": "assistant", "content": chat_history_prompt},
        {"role": "user", "content": question},
    ]

    print(message)

    response = chat_model.chat.completions.create(
                model='gpt-4o-mini',
                messages=message,
                temperature=0.7,
            )
  
    response_str = response.choices[0].message.content

    return response_str

if __name__ == "__main__":
    while True:
        user_input = input("\nEnter your question or type 'exit' to quit: ")
        if user_input.lower() == 'exit':
            print("Goodbye")
            break 
        chat_history.append(('human', user_input))
        response = get_response_from_llm(user_input)
        chat_history.append(('ai', response))
        print(response)

I hope you can successfully run the code!

Python Environment and Dependencies#

Install Python or Anaconda Environment Manager#

Create a Python Environment#

Activate the Environment and Install Dependencies#

Activate the Environment#

Install Relevant Dependencies#

Create a Python Script#

Prepare Your OpenAI API#

Start Coding#