CM5如何安装网卡驱动？
-
@jack_admin https://network.nvidia.com/products/ethernet-drivers/linux/mlnx_en/
The NVIDIA ConnectX adapter family can run Ethernet traffic simultaneously on two ports Single software stack that operates across all available NVIDIA Ethernet devices and configurations such as mem-free, up to 400Gb/s, and PCI Express modes 3.0, 4.0 and 5.0 Support for HPC applications for scientific research and AI Support for data center applications such as Oracle 11g/10g RAC, IBM DB2, Financial services applications such as IBM WebSphere LLM, Red Hat MRG, NYSE Data Fabric -
@Jachyn said in CM5如何安装网卡驱动？:
@大法师 大师 内核已经编译了 编译之后进行安装 又报如下的错
更新内核源码编译目录链接（假定源码目录在/home/coolpi/kernel）
sudo ln -sf /home/coolpi/kernel /lib/modules/5.10.110/build sudo ln -sf /home/coolpi/kernel /lib/modules/5.10.110/sourceMLNX_EN_SRC-debian-23.04-1.1.3.0.tgz
MLNX_EN_SRC-debian-5.8-3.0.7.0.tgz
解压网卡驱动源码，编译coolpi@coolpi:~/MLNX_EN_SRC-23.04-1.1.3.0$ sudo ./install.pl Logs dir: /tmp/OFED.80654.logs General log file: /tmp/OFED.80654.logs/general.log Below is the list of OFED packages that you have chosen (some may have been added by the installer due to package dependencies): ofed-scripts mlnx-tools mlnx-ofed-kernel-utils mlnx-ofed-kernel-modules Checking SW Requirements... One or more required packages for installing OFED-internal are missing. Attempting to install the following missing packages: swig make python3 chrpath gcc quilt debhelper dh-python python3-distutils This program will install the OFED package on your machine. Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed. Those packages are removed due to conflicts with OFED, do not reinstall them. Removing old packages get_requires: libibverbs1 is required by qemu-system-arm, but qemu-system-arm won't be removed. get_requires: librdmacm1 is required by qemu-system-arm, but qemu-system-arm won't be removed. Building packages Building DEB for ofed-scripts-23.04 (ofed-scripts)... Running /usr/bin/dpkg-buildpackage -us -uc Installing ofed-scripts-23.04... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/MLNX_EN_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/ofed-scripts_23.04-OFED.23.04.1.1.3_arm64.deb' Building DEB for mlnx-tools-23.04 (mlnx-tools)... Running /usr/bin/dpkg-buildpackage -us -uc Installing mlnx-tools-23.04... Running /usr/bin/dpkg -i --force-confmiss '/home/coolpi/MLNX_EN_SRC-23.04-1.1.3.0/DEBS/ubuntu22.04/aarch64/mlnx-tools_23.04-0.2304053_arm64.deb' Building DEB for mlnx-ofed-kernel-utils-23.04 (mlnx-ofed-kernel)... -W- --with-mlx5-ipsec is enabled Running /usr/bin/dpkg-buildpackage -us -uc -Pmodules ... -
@大法师 更新源码目录链接显示如下错误
ln: /lib/modules/5.10.110/build/kernel: cannot overwrite directory ln: /lib/modules/5.10.110/source/kernel: cannot overwrite directory -
@Jachyn said in CM5如何安装网卡驱动？:
@大法师 更新源码目录链接显示如下错误
ln: /lib/modules/5.10.110/build/kernel: cannot overwrite directory ln: /lib/modules/5.10.110/source/kernel: cannot overwrite directorysudo unlink /lib/modules/5.10.110/build sudo unlink /lib/modules/5.10.110/source然后再操作。编译可能有错误，你先排查下
-
@大法师 编译内核使用命令
make Image -j4，然后编译模块make modules -j4，并没有报错，内核make modules_install也是成功的
当前日志显示coolpi@coolpi:~/Downloads/mlnx-en-23.04-0.5.3.3-ubuntu22.04-aarch64$ sudo ./install --force-dkms Logs dir: /tmp/mlnx-en.6425.logs General log file: /tmp/mlnx-en.6425.logs/general.log Below is the list of mlnx-en packages that you have chosen (some may have been added by the installer due to package dependencies): ofed-scripts mlnx-tools mlnx-en-utils mlnx-en-dkms mstflint This program will install the mlnx-en package on your machine. Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed. Those packages are removed due to conflicts with mlnx-en, do not reinstall them. Do you want to continue?[y/N]:y Checking SW Requirements... Removing old packages... Uninstalling the previous version of mlnx-en Installing new packages Installing ofed-scripts-23.04... Installing mlnx-tools-23.04... Installing mlnx-en-utils-23.04... Installing mlnx-en-dkms-23.04... Installing mstflint-4.16.1... Selecting previously unselected package mlnx-fw-updater. (Reading database ... 196397 files and directories currently installed.) Preparing to unpack .../mlnx-fw-updater_23.04-0.5.3.3_arm64.deb ... Unpacking mlnx-fw-updater (23.04-0.5.3.3) ... Setting up mlnx-fw-updater (23.04-0.5.3.3) ... Initializing... Attempting to perform Firmware update... Querying Mellanox devices firmware ... Device #1: ---------- Device Type: ConnectX4LX Part Number: MCX4121A-ACA_Ax Description: ConnectX-4 Lx EN network interface card; 25GbE dual-port SFP28; PCIe3.0 x8; ROHS R6 PSID: MT_2420110034 PCI Device Name: 0001:11:00.0 Base MAC: 98039b2ba640 Versions: Current Available FW 14.22.1002 14.32.1010 PXE 3.5.0403 3.6.0502 UEFI 14.15.0019 14.25.0017 Status: Update required --------- Found 1 device(s) requiring firmware update... Device #1: Updating FW ... Done Restart needed for updates to take effect. Log File: /tmp/C3m3Iz3_P3 Real log file: /tmp/mlnx-en.6425.logs/fw_update.log Device (0001:11:00.0): 0001:11:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 2.5GT/s Device (0001:11:00.1): 0001:11:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 2.5GT/s Installation passed successfully To load the new driver, run: /etc/init.d/mlnx-en.d restart -
@Jachyn 好
-
coolpi@coolpi:~/Desktop$ /etc/init.d/mlnx-en.d restart Unloading NIC driver: [ OK ] Loading Mellanox MLX5 NIC driver: [FAILED] Loading NIC driver: [FAILED] Please run /usr/sbin/sysinfo-snapshot.py to collect the debug information and open an issue in the http://support.mellanox.com/SupportWeb/service_center/SelfService -
@Jachyn 加 sudo 执行试下
-
coolpi@coolpi:~/Desktop$ sudo /etc/init.d/mlnx-en.d restart Unloading NIC driver: [ OK ] Loading Mellanox MLX5 NIC driver: [FAILED] Loading NIC driver: [FAILED] Please run /usr/sbin/sysinfo-snapshot.py to collect the debug information and open an issue in the http://support.mellanox.com/SupportWeb/service_center/SelfService -
coolpi@coolpi:~/Downloads/mlnx-en-5.9-0.5.6.0-ubuntu22.04-aarch64$ sudo ./install --add-kernel-support Note: This program will create mlnx-en TGZ for ubuntu22.04 under /tmp/mlnx-en-5.9-0.5.6.0-5.10.110 directory. See log file /tmp/mlnx-en-5.9-0.5.6.0-5.10.110/mlnx_iso.109096_logs/mlnx_ofed_iso.109096.log Checking if all needed packages are installed... Building mlnx-en DEBS . Please wait... ERROR: Failed executing "MLNX_EN_SRC-5.9-0.5.6.0/install.pl --tmpdir /tmp/mlnx-en-5.9-0.5.6.0-5.10.110/mlnx_iso.109096_logs --eth-only --kernel 5.10.110 --kernel-sources /lib/modules/5.10.110/build --builddir /tmp/mlnx-en-5.9-0.5.6.0-5.10.110/mlnx_iso.109096 --without-dkms --without-debug-symbols --build-only --distro ubuntu22.04" ERROR: See /tmp/mlnx-en-5.9-0.5.6.0-5.10.110/mlnx_iso.109096_logs/mlnx_ofed_iso.109096.log Failed to build mlnx-en for 5.10.110是不是这系统就不支持这个驱动呢
-
@Jachyn said in CM5如何安装网卡驱动？:
coolpi@coolpi:~/Desktop$ sudo /etc/init.d/mlnx-en.d restart Unloading NIC driver: [ OK ] Loading Mellanox MLX5 NIC driver: [FAILED] Loading NIC driver: [FAILED] Please run /usr/sbin/sysinfo-snapshot.py to collect the debug information and open an issue in the http://support.mellanox.com/SupportWeb/service_center/SelfService执行这个之后，系统日志输出(dmesg)分析下错误原因。
-
@george 解决了这个驱动的问题，很感谢，但是我现在遇到了一个新的问题
我发现使用网卡后仍是pci进行操作的网卡，这个我该如何解决？ -
我是不是需要去内核修改一些相关的配置？
-
@Jake 有什么问题？网口节点出来了没
-
@jack_admin 用ifconfig/ip addr等命令查看并没有网卡，只有lspci | grep Eth可以看到有相关网卡pcie信息，但是查看pcie日志，并没有更换为网卡驱动
-
CMD: mlxup -u --log-on-update --ssl-certificate /tmp/SGKRmQP1We/mlxfwmanager_sriov_dis_aarch64_4115-dir/ca-bundle.crt --current-dir /opt/mellanox/mlnx-fw-updater/ -L /tmp/Nr5DC9bSSF -y -d 0001:11:00.0 Querying Mellanox devices firmware ... Device #1: ---------- Device Type: ConnectX4LX Part Number: -- Description: PSID: PCI Device Name: 0001:11:00.0 Base MAC: N/A Versions: Current Available FW -- Status: Failed to open device --------- -E- Failed to query 0001:11:00.0 device, error : No such file or directory. MFE_ICMD_NOT_SUPPORTED EXIT_STATUS: 0安装网卡驱动后 显示如上的日志
-
coolpi@coolpi:~/Downloads/mlnx-en-5.8-1.0.1.1-ubuntu22.04-aarch64$ sudo ./install --force-dkms [sudo] password for coolpi: Logs dir: /tmp/mlnx-en.738979.logs General log file: /tmp/mlnx-en.738979.logs/general.log Below is the list of mlnx-en packages that you have chosen (some may have been added by the installer due to package dependencies): ofed-scripts mlnx-tools mlnx-en-utils mlnx-en-dkms mstflint This program will install the mlnx-en package on your machine. Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed. Those packages are removed due to conflicts with mlnx-en, do not reinstall them. Do you want to continue?[y/N]:y Checking SW Requirements... Removing old packages... Uninstalling the previous version of mlnx-en Installing new packages Installing ofed-scripts-5.8... Installing mlnx-tools-5.2.0... Installing mlnx-en-utils-5.8... Installing mlnx-en-dkms-5.8... Installing mstflint-4.16.1... Selecting previously unselected package mlnx-fw-updater. (Reading database ... 199866 files and directories currently installed.) Preparing to unpack .../mlnx-fw-updater_5.8-1.0.1.1_arm64.deb ... Unpacking mlnx-fw-updater (5.8-1.0.1.1) ... Setting up mlnx-fw-updater (5.8-1.0.1.1) ... Initializing... Attempting to perform Firmware update... Querying Mellanox devices firmware ... Device #1: ---------- Device Type: ConnectX4LX Part Number: MCX4121A-ACA_Ax Description: ConnectX-4 Lx EN network interface card; 25GbE dual-port SFP28; PCIe3.0 x8; ROHS R6 PSID: MT_2420110034 PCI Device Name: 0001:11:00.0 Base MAC: 98039b2ba640 Versions: Current Available FW 14.32.1010 14.32.1010 PXE 3.6.0502 3.6.0502 UEFI 14.25.0017 14.25.0017 Status: Up to date Log File: /tmp/DR3i76jLzy Real log file: /tmp/mlnx-en.738979.logs/fw_update.log Device (0001:11:00.0): 0001:11:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 2.5GT/s Device (0001:11:00.1): 0001:11:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 2.5GT/s Installation passed successfully To load the new driver, run: /etc/init.d/mlnx-en.d restart coolpi@coolpi:~/Downloads/mlnx-en-5.8-1.0.1.1-ubuntu22.04-aarch64$ /etc/init.d/mlnx-en.d restart Unloading NIC driver: [ OK ] Loading Mellanox MLX5 NIC driver: [FAILED] Loading NIC driver: [FAILED] Please run /usr/sbin/sysinfo-snapshot.py to collect the debug information and open an issue in the http://support.mellanox.com/SupportWeb/service_center/SelfService -
coolpi@coolpi:~/Downloads/mlnx-en-5.8-2.0.3.0-ubuntu22.04-aarch64$ sudo ./install --force-dkms [sudo] password for coolpi: Logs dir: /tmp/mlnx-en.3438.logs General log file: /tmp/mlnx-en.3438.logs/general.log Below is the list of mlnx-en packages that you have chosen (some may have been added by the installer due to package dependencies): ofed-scripts mlnx-tools mlnx-en-utils mlnx-en-dkms mstflint This program will install the mlnx-en package on your machine. Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed. Those packages are removed due to conflicts with mlnx-en, do not reinstall them. Do you want to continue?[y/N]:y Checking SW Requirements... Removing old packages... Uninstalling the previous version of mlnx-en Installing new packages Installing ofed-scripts-5.8... Installing mlnx-tools-5.2.0... Installing mlnx-en-utils-5.8... Installing mlnx-en-dkms-5.8... Installing mstflint-4.16.1... Selecting previously unselected package mlnx-fw-updater. (Reading database ... 199866 files and directories currently installed.) Preparing to unpack .../mlnx-fw-updater_5.8-2.0.3.0_arm64.deb ... Unpacking mlnx-fw-updater (5.8-2.0.3.0) ... Setting up mlnx-fw-updater (5.8-2.0.3.0) ... Initializing... Attempting to perform Firmware update... Querying Mellanox devices firmware ... Device #1: ---------- Device Type: ConnectX4LX Part Number: MCX4121A-ACA_Ax Description: ConnectX-4 Lx EN network interface card; 25GbE dual-port SFP28; PCIe3.0 x8; ROHS R6 PSID: MT_2420110034 PCI Device Name: 0001:11:00.0 Base MAC: 98039b2ba640 Versions: Current Available FW 14.32.1010 14.32.1010 PXE 3.6.0502 3.6.0502 UEFI 14.25.0017 14.25.0017 Status: Up to date Log File: /tmp/05iafkfb8h Real log file: /tmp/mlnx-en.3438.logs/fw_update.log Device (0001:11:00.0): 0001:11:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 8GT/s Device (0001:11:00.1): 0001:11:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 8GT/s Installation passed successfully To load the new driver, run: /etc/init.d/mlnx-en.d restart coolpi@coolpi:~/Downloads/mlnx-en-5.8-2.0.3.0-ubuntu22.04-aarch64$ /etc/init.d/mlnx-en.d restart Unloading NIC driver: [ OK ] Loading Mellanox MLX5 NIC driver: [FAILED] Loading NIC driver: [FAILED] Please run /usr/sbin/sysinfo-snapshot.py to collect the debug information and open an issue in the http://support.mellanox.com/SupportWeb/service_center/SelfService coolpi@coolpi:~/Downloads/mlnx-en-5.8-2.0.3.0-ubuntu22.04-aarch64$ -
coolpi@coolpi:~/Downloads/mlnx-en-5.6-2.0.9.0-ubuntu22.04-aarch64$ sudo ./install --force-dkms [sudo] password for coolpi: Logs dir: /tmp/mlnx-en.514532.logs General log file: /tmp/mlnx-en.514532.logs/general.log Below is the list of mlnx-en packages that you have chosen (some may have been added by the installer due to package dependencies): ofed-scripts mlnx-tools mlnx-en-utils mlnx-en-dkms mstflint This program will install the mlnx-en package on your machine. Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed. Those packages are removed due to conflicts with mlnx-en, do not reinstall them. Do you want to continue?[y/N]:y Checking SW Requirements... Removing old packages... Uninstalling the previous version of mlnx-en Installing new packages Installing ofed-scripts-5.6... Installing mlnx-tools-5.2.0... Installing mlnx-en-utils-5.6... Installing mlnx-en-dkms-5.6... Installing mstflint-4.16.1... Selecting previously unselected package mlnx-fw-updater. (Reading database ... 199769 files and directories currently installed.) Preparing to unpack .../mlnx-fw-updater_5.6-2.0.9.0_arm64.deb ... Unpacking mlnx-fw-updater (5.6-2.0.9.0) ... Setting up mlnx-fw-updater (5.6-2.0.9.0) ... Initializing... Attempting to perform Firmware update... Querying Mellanox devices firmware ... Device #1: ---------- Device Type: ConnectX4LX Part Number: MCX4121A-ACA_Ax Description: ConnectX-4 Lx EN network interface card; 25GbE dual-port SFP28; PCIe3.0 x8; ROHS R6 PSID: MT_2420110034 PCI Device Name: 0001:11:00.0 Base MAC: 98039b2ba640 Versions: Current Available FW 14.32.1010 14.32.1010 PXE 3.6.0502 3.6.0502 UEFI 14.25.0017 14.25.0017 Status: Up to date Log File: /tmp/07OcqgbJK0 Real log file: /tmp/mlnx-en.514532.logs/fw_update.log Device (0001:11:00.0): 0001:11:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 8GT/s Device (0001:11:00.1): 0001:11:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 8GT/s Installation passed successfully To load the new driver, run: /etc/init.d/mlnx-en.d restart coolpi@coolpi:~/Downloads/mlnx-en-5.6-2.0.9.0-ubuntu22.04-aarch64$ /etc/init.d/mlnx-en.d restart Unloading NIC driver: [ OK ] Loading Mellanox MLX5 NIC driver: [FAILED] Loading NIC driver: [FAILED] Please run /usr/sbin/sysinfo-snapshot.py to collect the debug information and open an issue in the http://support.mellanox.com/SupportWeb/service_center/SelfService -
coolpi@coolpi:~/Downloads/mlnx-en-5.7-1.0.2.0-ubuntu22.04-aarch64$ sudo ./install --force-dkms [sudo] password for coolpi: Logs dir: /tmp/mlnx-en.759187.logs General log file: /tmp/mlnx-en.759187.logs/general.log Below is the list of mlnx-en packages that you have chosen (some may have been added by the installer due to package dependencies): ofed-scripts mlnx-tools mlnx-en-utils mlnx-en-dkms mstflint This program will install the mlnx-en package on your machine. Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed. Those packages are removed due to conflicts with mlnx-en, do not reinstall them. Do you want to continue?[y/N]:y Checking SW Requirements... Removing old packages... Uninstalling the previous version of mlnx-en Installing new packages Installing ofed-scripts-5.7... Installing mlnx-tools-5.2.0... Installing mlnx-en-utils-5.7... Installing mlnx-en-dkms-5.7... Installing mstflint-4.16.1... Selecting previously unselected package mlnx-fw-updater. (Reading database ... 199851 files and directories currently installed.) Preparing to unpack .../mlnx-fw-updater_5.7-1.0.2.0_arm64.deb ... Unpacking mlnx-fw-updater (5.7-1.0.2.0) ... Setting up mlnx-fw-updater (5.7-1.0.2.0) ... Initializing... Attempting to perform Firmware update... Querying Mellanox devices firmware ... Device #1: ---------- Device Type: ConnectX4LX Part Number: MCX4121A-ACA_Ax Description: ConnectX-4 Lx EN network interface card; 25GbE dual-port SFP28; PCIe3.0 x8; ROHS R6 PSID: MT_2420110034 PCI Device Name: 0001:11:00.0 Base MAC: 98039b2ba640 Versions: Current Available FW 14.32.1010 14.32.1010 PXE 3.6.0502 3.6.0502 UEFI 14.25.0017 14.25.0017 Status: Up to date Log File: /tmp/FFuM6_xJPq Real log file: /tmp/mlnx-en.759187.logs/fw_update.log Device (0001:11:00.0): 0001:11:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 8GT/s Device (0001:11:00.1): 0001:11:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx] Link Width: x2 ( WARNING - device supports x8 ) PCI Link Speed: 8GT/s Installation passed successfully To load the new driver, run: /etc/init.d/mlnx-en.d restart coolpi@coolpi:~/Downloads/mlnx-en-5.7-1.0.2.0-ubuntu22.04-aarch64$ /etc/init.d/mlnx-en.d restart Unloading NIC driver: [ OK ] Loading Mellanox MLX5 NIC driver: [FAILED] Loading NIC driver: [FAILED] Please run /usr/sbin/sysinfo-snapshot.py to collect the debug information and open an issue in the http://support.mellanox.com/SupportWeb/service_center/SelfService
