iSCSI issue in Server 2016 Hyper-V cluster

G

GCHamby

Greetings all. I have a customer with a two node Server 2016 Hyper-V cluster. System was installed in Dec 2017 and ran fine for probably 7 to 8 months when we began experiencing issues. Hosts are identical Dell R830s each with two dual port QLogic 57810 10Gb adapters. The adapters are using DAC cables to connect to the respective ports of an EMC Unity 300 SAN. Each adapter has one copper cable going to one port of a storage processor on the SAN as depicted in the attached JPG.

The hosts were recently updated with the latest firmware, drivers, and OS updates for server 2016 in an attempt to rectify the intermittent iSCSI timeouts we're experiencing.

Users began noticing that VMs will sometimes "freeze" or become un responsive for a minute or two and then resume. We occasionally get application level errors about I/O that completed but "took an abnormally long time".

The Windows system event log will often record iSCSIprt errors event ID 9 that the "Target did not respond in time for a SCSI request".

We get these on both hosts and to two different LUNs on the Unity. Typically the events occur in bursts all with the same timestamp and sometimes a few events are recorded and sometimes dozens.

Since everything has now been updated to the latest available firmware and drivers we're not sure where to go from here. MPIO is installed on the hosts but the targets were not configured for MPIO because there is no switch in the mix. These hosts are direct connected to storage.

If anyone has seen behavior like this or has a suggestion for additional troubleshooting I'd welcome your comments.

Thanks.

1397386.jpg

Continue reading...
 
Back
Top Bottom