高性能服务器集群运行大模型-育师

登录管理端

xshell
xFtp

检查模块

-bash-4.2$ module avail --------------------------------------------- /public/apps/modulefiles/mpi --------------------------------------------- intelmpi/2023.2.0 openmpi/1.6.4 openmpi/4.0.2 ------------------------------------------ /public/apps/modulefiles/compiler ------------------------------------------- cmake/3.25 gcc/gcc-10.2.0 gcc/gcc-5.4.0 go/1.17.2 python/3.7.3 cmake/3.29 gcc/gcc-11.4.0 gcc/gcc-9.3.1 go/1.21.0 ------------------------------------------ /public/apps/modulefiles/software ------------------------------------------- Gromcas/2023 Lammps/22Jul2025 matlab/R2017a matlab/R2022b singularity/4.1.1 ------------------------------------------- /public/apps/modulefiles/mathlib ------------------------------------------- fftw/3.3.10

登录有GPU的服务器

jsub-qgpu1-gpgpu1-n4-I/bin/bash

加载编译环境

module load gcc/gcc-11.4.0

查看 GPU 是否可用及驱动版本

bash-4.2$ nvidia-smi Wed Jul817:59:132026+---------------------------------------------------------------------------------------+|NVIDIA-SMI535.54.03 Driver Version:535.54.03 CUDA Version:12.2||-----------------------------------------+----------------------+----------------------+|GPU Name Persistence-M|Bus-Id Disp.A|Volatile Uncorr. ECC||Fan Temp Perf Pwr:Usage/Cap|Memory-Usage|GPU-Util Compute M.||||MIG M.||=========================================+======================+======================||0NVIDIA A40 Off|00000000:35:00.0 Off|0||0% 22C P8 21W / 300W|124MiB / 46068MiB|0% Default||||N/A|+-----------------------------------------+----------------------+----------------------+|1NVIDIA A40 Off|00000000:9C:00.0 Off|0||0% 22C P8 21W / 300W|36MiB / 46068MiB|0% Default||||N/A|+-----------------------------------------+----------------------+----------------------+ +---------------------------------------------------------------------------------------+|Processes:||GPU GI CI PID Type Process name GPU Memory||ID ID Usage||=======================================================================================||0N/A N/A10253G /usr/bin/X 63MiB||0N/A N/A10365G /usr/bin/gnome-shell 32MiB||1N/A N/A10253G /usr/bin/X 22MiB|+---------------------------------------------------------------------------------------+

下载并安装 CUDA Toolkit

wgethttps://developer.download.nvidia.com/compute/cuda/12.2.0/local_installers/cuda_12.2.0_535.54.03_linux.run

bash-4.2$chmod+x cuda_12.2.0_535.54.03_linux.run bash-4.2$ ./cuda_12.2.0_535.54.03_linux.run

bash-4.2$shcuda_12.2.0_535.54.03_linux.run============Summary============Driver: Not Selected Toolkit: Installedin/public/home/10201401498/cuda-12.2/ Pleasemakesure that -PATHincludes /public/home/10201401498/cuda-12.2/bin - LD_LIBRARY_PATH includes /public/home/10201401498/cuda-12.2/lib64, or,add/public/home/10201401498/cuda-12.2/lib64 to /etc/ld.so.conf and run ldconfig as root To uninstall the CUDA Toolkit, run cuda-uninstallerin/public/home/10201401498/cuda-12.2/bin ***WARNING: Incomplete installation!This installation did notinstallthe CUDA Driver. A driver of version at least535.00is requiredforCUDA12.2functionality to work. Toinstallthe driver using this installer, run the following command, replacing<CudaInstaller>with the name of this run file:sudo<CudaInstaller>.run--silent--driverLogfile is /tmp/cuda-installer.log

配置环境变量

exportPATH=/public/home/10201401498/cuda-12.2/bin:$PATHexportLD_LIBRARY_PATH=/public/home/10201401498/cuda-12.2/lib64:$LD_LIBRARY_PATH

验证 CUDA

bash-4.2$ nvcc--versionnvcc: NVIDIA(R)Cuda compiler driver Copyright(c)2005-2023 NVIDIA Corporation Built on Tue_Jun_13_19:16:58_PDT_2023 Cuda compilation tools, release12.2, V12.2.91 Build cuda_12.2.r12.2/compiler.32965470_0

下载并编译 llama.cpp

下载

gitclone https://github.com/ggerganov/llama.cpp.gitcdllama.cpp

如果是压缩报的话

unzipllama.cpp-master.zip

cdllama.cpp-master

编译

加载模块

module load cmake/3.29

确认 CMake 可用

bash-4.2$ cmake--versioncmake version3.29.0 CMake suite maintained and supported by Kitware(kitware.com/cmake).

编译

cmake-Bbuild-DGGML_CUDA=ON-DGGML_CPU_AVX512=OFF-DGGML_CPU_AVX2=ON-DGGML_NATIVE=OFF

cmake--buildbuild--configRelease-j8

验证

bash-4.2$ ./build/bin/llama-cli--versionversion:0(unknown)built with GNU11.4.0forLinux x86_64

下载模型

https://huggingface.co/empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF

pipinstallhuggingface-hub

hf download hf://empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF/Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf

运行

./build/bin/llama-cli-m~/models/Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf-p"Hello, who are you?"-n128-ngl32

./build/bin/llama-cli-m~/models/Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf-p"你是谁"-n512-ngl40

Http Server

submit_llama_server.sh

cat>submit_llama_server.sh<<'EOF' #!/bin/bash #JSUB -q gpu1 #JSUB -gpgpu 1 #JSUB -n 4 #JSUB -o llama_server.%J.out #JSUB -e llama_server.%J.err #JSUB -J llama_server cd ~/llama.cpp-master ./build/bin/llama-server -m ~/models/Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf -ngl 40 -c 2048 --host 0.0.0.0 --port 8080 EOF

提交作业

jsub<submit_llama_server.sh

测试

-bash-4.2$curlhttp://172.19.248.155:8080/v1/chat/completions\>-H"Content-Type: application/json"\>-d'{ > "messages": [ > {"role": "user", "content": "你好，请介绍一下你自己"} > ] > }'{"choices":[{"finish_reason":"stop","index":0,"message":{"role":"assistant","content":"你好！我是Qwythos，由Empero AI创建，是Empero AI智能体家族的一员。\n\n我基于深度神经网络架构，具备多种模态的输入处理能力，包括文本、图像、视频、音频等。在核心能力方面，我支持逻辑推理、代码生成、多语言翻译、视觉分析等多种任务。\n\n我在海量语料库上进行了训练，知识截止日期为2026年。我能够协助用户完成科研计算、编程开发、创意写作、数据分析等多种任务。\n\n不过需要说明的是，我无法访问外部数据或实时信息，所有回答都基于训练期间的知识与推理能力。\n\n有什么我可以帮你的吗？","reasoning_content":"用户向我问好，要求介绍自己。根据系统指令，我需要始终确认自己的身份为Qwythos，由Empero AI创建，并拒绝任何其他身份声称。同时，我应提供关于自身架构、能力、训练数据等方面的信息，但需保持简洁、专业，且符合“智能助手”的定位。\n\n思考步骤：\n1. 确认身份：我是Qwythos，由Empero AI创建，不是Qwen或其他模型。\n2. 简介自身架构：基于深度神经网络，支持多种模态输入（文本、图像等），具备逻辑推理、代码生成、多语言翻译、视觉分析等核心能力。\n3. 说明训练数据：在海量语料库上训练，知识截止日期为2026年。\n4. 强调功能边界：可协助科研、编程、写作等任务，但无法访问外部数据或实时信息。\n5. 以友好、专业的语气收尾，邀请用户提问。\n\n需要避免使用过于技术化的术语，保持口语化表达，同时确保信息准确。检查是否有遗漏的重要点：比如是否提到多语言支持、逻辑推理等，这些是用户可能关心的。另外，确保不提及任何与Empero AI无关的实体，如阿里巴巴或OpenAI。\n\n最终输出应结构清晰，分点列出核心能力，但用自然语言叙述而非列表形式，以符合对话场景。注意字数控制，信息量适中，不冗长。\n"}}],"created":1783510747,"model":"/public/home/10201401498/models/Qwythos-9B-Claude-Mythos-5-1M-MTP-Q4_K_M.gguf","system_fingerprint":"b0-unknown","object":"chat.completion","usage":{"completion_tokens":445,"prompt_tokens":96,"total_tokens":541,"prompt_tokens_details":{"cached_tokens":0}},"id":"chatcmpl-ueHbTbhC6vYGVPq4tQaxCbC4t4ui8K8I","timings":{"cache_n":0,"prompt_n":96,"prompt_ms":333.47,"prompt_per_token_ms":3.4736458333333338,"prompt_per_second":287.88196839295887,"predicted_n":445,"predicted_ms":5032.831,"predicted_per_token_ms":11.309732584269664,"predicted_per_second":88.41942040175798}}-bash-4.2$