N
next40
Good day, i have 4-node Cluster Based on Microsoft S2D tech, after configuration smb multichannel i see unstable speed between nodes.
Each server have HP Ethernet 10Gb 2-port 546SFP+ Adapter( Mellanox ConnectX-3 Pro Dual Port Adapter)
http://www.mellanox.com/blog/2015/0...working-for-high-performance-infrastructures/
Confuguration:
Set-SmbServerConfiguration -EnableMultiChannel $true -Confirm:$false
Set-SmbClientConfiguration -EnableMultiChannel $true -Confirm:$false
New-SmbMultichannelConstraint -InterfaceAlias RDMA-NIC1, RDMA-NIC2 -ServerName node1 -Confirm:$false
New-SmbMultichannelConstraint -InterfaceAlias RDMA-NIC1, RDMA-NIC2 -ServerName node2 -Confirm:$false
New-SmbMultichannelConstraint -InterfaceAlias RDMA-NIC1, RDMA-NIC2 -ServerName node3 -Confirm:$false
New-SmbMultichannelConstraint -InterfaceAlias RDMA-NIC1, RDMA-NIC2 -ServerName node4 -Confirm:$false
Restart-NetAdapter RDMA-NIC1,RDMA-NIC2
After Enabling i see performance degradation, example copy from boot drive to boot drive:
If i disable one port on adapter any try copying via individual nic, i see stable speed:
I trying to find problems on my system and found this article
The basics of SMB Multichannel, a feature of Windows Server 2012 and SMB 3.0
Event Viewer reports some errors:
The description for Event ID 30804 from source Microsoft-Windows-SMBClient cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
7
3221225996
13
\10.100.100.8
16
1
The handle is invalid
---------
The description for Event ID 30803 from source Microsoft-Windows-SMBClient cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
4
3221225653
28
fe80::1d46:e01c:b420:405b%10
16
1
The handle is invalid
------
A network connection was disconnected.
Server name: \fe80::1d46:e01c:b420:405bRdma
Server address: 10.100.100.108:445
Connection type: Rdma
Guidance:
This indicates that the client's connection to the server was disconnected.
Frequent, unexpected disconnects when using an RDMA over Converged Ethernet (RoCE) adapter may indicate a network misconfiguration. RoCE requires Priority Flow Control (PFC) to be configured for every host, switch and router on the RoCE network. Failure to properly configure PFC will cause packet loss, frequent disconnects and poor performance.
IPv6 Disabled on the all nodes via ( Get-NetAdapter | Disable-NetAdapterBinding -ComponentID ms_tcpip6 )
Please help to troubleshoot this problem!
Continue reading...
Each server have HP Ethernet 10Gb 2-port 546SFP+ Adapter( Mellanox ConnectX-3 Pro Dual Port Adapter)
http://www.mellanox.com/blog/2015/0...working-for-high-performance-infrastructures/
Confuguration:
Set-SmbServerConfiguration -EnableMultiChannel $true -Confirm:$false
Set-SmbClientConfiguration -EnableMultiChannel $true -Confirm:$false
New-SmbMultichannelConstraint -InterfaceAlias RDMA-NIC1, RDMA-NIC2 -ServerName node1 -Confirm:$false
New-SmbMultichannelConstraint -InterfaceAlias RDMA-NIC1, RDMA-NIC2 -ServerName node2 -Confirm:$false
New-SmbMultichannelConstraint -InterfaceAlias RDMA-NIC1, RDMA-NIC2 -ServerName node3 -Confirm:$false
New-SmbMultichannelConstraint -InterfaceAlias RDMA-NIC1, RDMA-NIC2 -ServerName node4 -Confirm:$false
Restart-NetAdapter RDMA-NIC1,RDMA-NIC2
After Enabling i see performance degradation, example copy from boot drive to boot drive:
If i disable one port on adapter any try copying via individual nic, i see stable speed:
I trying to find problems on my system and found this article
The basics of SMB Multichannel, a feature of Windows Server 2012 and SMB 3.0
Event Viewer reports some errors:
The description for Event ID 30804 from source Microsoft-Windows-SMBClient cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
7
3221225996
13
\10.100.100.8
16
1
The handle is invalid
---------
The description for Event ID 30803 from source Microsoft-Windows-SMBClient cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.
If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
4
3221225653
28
fe80::1d46:e01c:b420:405b%10
16
1
The handle is invalid
------
A network connection was disconnected.
Server name: \fe80::1d46:e01c:b420:405bRdma
Server address: 10.100.100.108:445
Connection type: Rdma
Guidance:
This indicates that the client's connection to the server was disconnected.
Frequent, unexpected disconnects when using an RDMA over Converged Ethernet (RoCE) adapter may indicate a network misconfiguration. RoCE requires Priority Flow Control (PFC) to be configured for every host, switch and router on the RoCE network. Failure to properly configure PFC will cause packet loss, frequent disconnects and poor performance.
IPv6 Disabled on the all nodes via ( Get-NetAdapter | Disable-NetAdapterBinding -ComponentID ms_tcpip6 )
Please help to troubleshoot this problem!
Continue reading...