gpt4all配置

快速安装：

pip install gpt4all

代码示例：

from enum import Enum
from gpt4all import GPT4All

class ModelName(Enum):
    gpt4all_snoozy_q4_0     = "gpt4all-13b-snoozy-q4_0.gguf"
    orca_mini_3b_gguf2      = "orca-mini-3b-gguf2-q4_0.gguf"
    wizardlm_13b_v1_2_Q4    = "wizardlm-13b-v1.2.Q4_0.gguf"
    rift_coder              = "rift-coder-v0-7b-q4_0.gguf"
    incomplete_orca_2_13b   = "incomplete-orca-2-13b.Q4_0.gguf"
    mistral_7b_openorca     = "mistral-7b-openorca.gguf"


model_name = ModelName.gpt4all_snoozy_q4_0.value
model_path = f'C:\\OpenAi\\GPT4All\\Models';

print(model_name);
model = GPT4All(model_name=model_name, model_path=model_path);
output = model.generate("你的最后更新日期是什么时间？", temp=0)
print(output)
output = model.generate("你叫什么名字？", temp=0)
print(output)
output = model.generate("你会说中文吗？", temp=0)
print(output)

代码编译：

工具及环境: VSCode + Vulkan
Vulkan 下载地址

下载 gpt4all 代码

git clone --recurse-submodules https://github.com/nomic-ai/gpt4all.git
cd gpt4all/gpt4all-backend

编译：

cmake 下载地址

编译环境参考:

VSCode + vcpkg编译C-程序示例

将 VCPKG_ROOT 和 C:\VulkanSDK\1.3.275.0 指向实际地址

set "VCPKG_ROOT=D:\WorkProject\vcpkg\vcpkg"
set VULKAN_SDK=C:\VulkanSDK\1.3.275.0

set PATH=%VCPKG_ROOT%;%VULKAN_SDK%\Bin;%PATH%

cmake -B build

cmake --build build --parallel --config RelWithDebInfo
#这一步需要时间较长

安装Python包

cd ../../gpt4all-bindings/python
pip install -e .

如下所示：

D:\WorkProject\gpt4all\gpt4all-bindings\python>pip install -e .
Looking in indexes: http://mirrors.aliyun.com/pypi/simple/, https://pypi.python.org/simple
Obtaining file:///D:/WorkProject/gpt4all/gpt4all-bindings/python
  Preparing metadata (setup.py) ... done
Requirement already satisfied: requests in d:\python\python39\lib\site-packages (from gpt4all==2.3.0) (2.31.0)
Requirement already satisfied: tqdm in d:\python\python39\lib\site-packages (from gpt4all==2.3.0) (4.66.1)
Requirement already satisfied: charset-normalizer<4,>=2 in d:\python\python39\lib\site-packages (from requests->gpt4all==2.3.0) (3.2.0)
Requirement already satisfied: idna<4,>=2.5 in d:\python\python39\lib\site-packages (from requests->gpt4all==2.3.0) (2.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in d:\python\python39\lib\site-packages (from requests->gpt4all==2.3.0) (1.26.15)
Requirement already satisfied: certifi>=2017.4.17 in d:\python\python39\lib\site-packages (from requests->gpt4all==2.3.0) (2023.7.22)
Requirement already satisfied: colorama in d:\python\python39\lib\site-packages (from tqdm->gpt4all==2.3.0) (0.4.6)
Installing collected packages: gpt4all
  Attempting uninstall: gpt4all
    Found existing installation: gpt4all 2.2.1.post1
    Uninstalling gpt4all-2.2.1.post1:
      Successfully uninstalled gpt4all-2.2.1.post1
  Running setup.py develop for gpt4all
Successfully installed gpt4all-2.3.0

示例：

from gpt4all import GPT4All
model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf")
output = model.generate("The capital of France is ", max_tokens=3)
print(output)

GPU Usage

from gpt4all import GPT4All
model = GPT4All("orca-mini-3b-gguf2-q4_0.gguf", device='gpu') # device='amd', device='intel'
output = model.generate("The capital of France is ", max_tokens=3)
print(output)

遇到问题：

gpt4all-backend cmake -B build 错误

文件目录 gpt4all\gpt4all-backend> 下 运行 cmake -B build 时报以下错误

        ?
Call Stack (most recent call first):
  llama.cpp-mainline/kompute/CMakeLists.txt:118 (check_vulkan_version)


-- Configuring incomplete, errors occurred!

解决办法1：

通过将 -DKOMPUTE_OPT_DISABLE_VULKAN_VERSION_CHECK=ON 传递给 CMake（CI 执行此操作）来禁用此检查
cmake -B build -DKOMPUTE_OPT_DISABLE_VULKAN_VERSION_CHECK=ON
运行结果

D:\WorkProject\gpt4all\gpt4all-backend>cmake -B build -DKOMPUTE_OPT_DISABLE_VULKAN_VERSION_CHECK=ON
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.22621.
-- Interprocedural optimization support detected
-- Kompute found
-- General purpose GPU compute framework built on Vulkan
-- =======================================================
--   KOMPUTE_OPT_LOG_LEVEL: Critical
--   KOMPUTE_OPT_USE_SPDLOG: OFF
--   KOMPUTE_OPT_DISABLE_VK_DEBUG_LAYERS: ON
--   KOMPUTE_OPT_DISABLE_VULKAN_VERSION_CHECK: ON
--   KOMPUTE_OPT_BUILD_SHADERS: OFF
--   KOMPUTE_OPT_USE_BUILT_IN_SPDLOG: ON
--   KOMPUTE_OPT_SPDLOG_ASYNC_MODE: OFF
--   KOMPUTE_OPT_USE_BUILT_IN_FMT: ON
--   KOMPUTE_OPT_USE_BUILT_IN_VULKAN_HEADER: ON
--   KOMPUTE_OPT_BUILT_IN_VULKAN_HEADER_TAG: v1.3.231
-- =======================================================
-- Version: 10.0.0
-- Build type:
-- Using log level Critical
-- shaderop_scale.h generating SHADEROP_SCALE_H
-- shaderop_scale_8.h generating SHADEROP_SCALE_8_H
-- shaderop_add.h generating SHADEROP_ADD_H
-- shaderop_addrow.h generating SHADEROP_ADDROW_H
-- shaderop_mul.h generating SHADEROP_MUL_H
-- shaderop_silu.h generating SHADEROP_SILU_H
-- shaderop_relu.h generating SHADEROP_RELU_H
-- shaderop_gelu.h generating SHADEROP_GELU_H
-- shaderop_softmax.h generating SHADEROP_SOFTMAX_H
-- shaderop_norm.h generating SHADEROP_NORM_H
-- shaderop_rmsnorm.h generating SHADEROP_RMSNORM_H
-- shaderop_diagmask.h generating SHADEROP_DIAGMASK_H
-- shaderop_mul_mat_mat_f32.h generating SHADEROP_MUL_MAT_MAT_F32_H
-- shaderop_mul_mat_f16.h generating SHADEROP_MUL_MAT_F16_H
-- shaderop_mul_mat_q8_0.h generating SHADEROP_MUL_MAT_Q8_0_H
-- shaderop_mul_mat_q4_0.h generating SHADEROP_MUL_MAT_Q4_0_H
-- shaderop_mul_mat_q4_1.h generating SHADEROP_MUL_MAT_Q4_1_H
-- shaderop_mul_mat_q6_k.h generating SHADEROP_MUL_MAT_Q6_K_H
-- shaderop_getrows_f16.h generating SHADEROP_GETROWS_F16_H
-- shaderop_getrows_q4_0.h generating SHADEROP_GETROWS_Q4_0_H
-- shaderop_getrows_q4_1.h generating SHADEROP_GETROWS_Q4_1_H
-- shaderop_getrows_q6_k.h generating SHADEROP_GETROWS_Q6_K_H
-- shaderop_rope_f16.h generating SHADEROP_ROPE_F16_H
-- shaderop_rope_f32.h generating SHADEROP_ROPE_F32_H
-- shaderop_cpy_f16_f16.h generating SHADEROP_CPY_F16_F16_H
-- shaderop_cpy_f16_f32.h generating SHADEROP_CPY_F16_F32_H
-- shaderop_cpy_f32_f16.h generating SHADEROP_CPY_F32_F16_H
-- shaderop_cpy_f32_f32.h generating SHADEROP_CPY_F32_F32_H
-- CMAKE_SYSTEM_PROCESSOR: AMD64
-- CMAKE_GENERATOR_PLATFORM:
-- x86 detected
-- Configuring ggml implementation target llama-mainline-default in D:/WorkProject/gpt4all/gpt4all-backend/llama.cpp-mainline
-- x86 detected
-- Configuring model implementation target llamamodel-mainline-default
-- Configuring model implementation target gptj-default
-- Configuring ggml implementation target llama-mainline-avxonly in D:/WorkProject/gpt4all/gpt4all-backend/llama.cpp-mainline
-- x86 detected
-- Configuring model implementation target llamamodel-mainline-avxonly
-- Configuring model implementation target gptj-avxonly
-- Configuring done (15.6s)
-- Generating done (0.2s)
-- Build files have been written to: D:/WorkProject/gpt4all/gpt4all-backend/build

解决办法2：

注释掉 gpt4all-backend/CMakeLists.txt 中的 LLAMA_KOMPUTE 行来完全禁用 Vulkan 依赖项，如下所示：

if(NOT APPLE)
  # set(LLAMA_KOMPUTE YES)
endif()

问题3：

gpt4all\gpt4all-backend>cmake -B build -DKOMPUTE_OPT_DISABLE_VULKAN_VERSION_CHECK=ON
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.22621.
-- Interprocedural optimization support detected
-- Kompute found
CMake Error at C:/Program Files/CMake/share/cmake-3.29/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found
  version "")
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.29/Modules/FindPackageHandleStandardArgs.cmake:600 (_FPHSA_FAILURE_MESSAGE)
  C:/Program Files/CMake/share/cmake-3.29/Modules/FindVulkan.cmake:594 (find_package_handle_standard_args)
  llama.cpp.cmake:170 (find_package)
  CMakeLists.txt:42 (include)


-- Configuring incomplete, errors occurred!

设置环境变量，
注：在运行 cmake 命令的终端窗口执行以下命令，设置成实际目录
set VULKAN_SDK=C:\VulkanSDK\1.3.275.0
set PATH=%VULKAN_SDK%\Bin;%PATH%

本地构建故障

如果您使用的是 Windows 并使用 MinGW 工具链进行编译，您可能会遇到如下错误：

self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module '<...>\gpt4all\gpt4all-bindings\python\gpt4all\llmodel_DO_NOT_MODIFY\build\llmodel.dll' (or one of its dependencies). Try using the full path with constructor syntax.

原因是找不到依赖项，
libgcc_s_seh-1.dll,
libstdc++-6.dll
libwinpthread-1.dll
需要将上面的3个文件复制到 python 可以找到的文件夹中最好是在 libllmodel.dll 所在的目录

参考目录：
gpt4all\gpt4all-bindings\python\gpt4all\llmodel_DO_NOT_MODIFY\build\

<Python安装目录>\Lib\site-packages\gpt4all\llmodel_DO_NOT_MODIFY\build\

参考链接