求助, python3.11 调用 chatGLM2_6b 模型

16次阅读

共计 2346 个字符,预计需要花费 6 分钟才能阅读完成。

求助,python3.11 调用 chatGLM2_6b 模型,单纯跑个 ChatGLM2-6B 的本地模型,但是就这只要已发消息就卡死了,难道是本地配置不够吗,求教求教,感谢

设备:macbook pro M2 32G

main.py

import tkinter as tk

from tkinter import scrolledtext
from gpt.OpenAI.openAI import get_openai_response
from gpt.ChatGLM.chatglm_client import get_chatGLM2_6b_response

def send_message(event=None):
    user_input = input_text.get()
    chat_history.insert(tk.END, f"You: {user_input}n")
    input_text.set("")

    response = get_chatGLM2_6b_response(user_input)
    chat_history.insert(tk.END, f"Bot: {response}n")


# 创建主窗口
root = tk.Tk()
root.title("Chat with OpenAI")

# 创建聊天记录文本框
chat_history = scrolledtext.ScrolledText(root, wrap=tk.WORD)
chat_history.pack(padx=10, pady=10, fill=tk.BOTH, expand=True)

# 创建输入框和发送按钮
input_text = tk.StringVar()
entry_box = tk.Entry(root, textvariable=input_text, width=50)
entry_box.pack(padx=10, pady=5, side=tk.LEFT, expand=True)
entry_box.bind('', send_message)

send_button = tk.Button(root, text="Send", command=send_message)
send_button.pack(padx=10, pady=5, side=tk.RIGHT)

# 运行主循环
root.mainloop()

chatglm_client.py

import torch

from transformers import AutoTokenizer, AutoModel

# 确定设备是否支持 mps
device = "mps" if torch.backends.mps.is_available() else "cpu"

# 提前加载模型
print("Device:", device)
print("Loading tokenizer...")
tokenizer = AutoTokenizer.from_pretrained("./modles/chatglm-6b", trust_remote_code=True)
print("Loading model...")
model = AutoModel.from_pretrained("./modles/chatglm-6b", trust_remote_code=True).half().to(device)
print("Model loaded.")

def get_chatGLM2_6b_response(prompt):
    print(prompt)
    inputs = tokenizer(prompt, return_tensors="pt").to(device)
    attention_mask = inputs.attention_mask.to(device) if 'attention_mask' in inputs else None
    
    print("Generating response...")
    outputs = model.generate(inputs.input_ids, attention_mask=attention_mask, max_length=50)
    
    print("Decoding response...")
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    print(response)
    
    return response

输出

启动后输出到’Model loaded.‘这一行
输入后输出到’Generating response…‘这一行就不动了。。。。
python main.py
Device: mps
Loading tokenizer...
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Loading model...
Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Loading checkpoint shards:   0%|                                                | 0/8 [00:00

所以想请教这种情况是不是系统内存不够之类的原因

正文完
 0