Sorry for the response delay.

1. Because 10GbE is so fast, I would use SET for all adapters and then use vNICs for iSCSI
2. If you have a use for DCB, sure. Cluster traffic is SMB so you would have to create a priority level for SMB.
3. Don’t overthink it. 10GbE is fast. Anything beyond a basic connection is probably over-engineering.

Your posts are really helpful, I read everything related to this and new hyperv clust and then some more and I have a few question:

I’m setting up a new 3 cluster environment, each 1 has 4X10 10gbe 2X1gbe(1 disabled and 1 for DMZ) and compelent SCv3020 storage. 2 new 10gbe switches (the rest are old 1gbe).

1) All of the 10gbe ports are connected to the 10gbe switches (I wanted to be able to use the 10gbe for the VMs as well). Should I use SET with all the 4 adapters and create adapters for ISCSI and the cluster or should I use 2 with SETteam and keep the 2 ISCSI separate?

2) Should I enable DCB? Can it work with normal cluster data on the switches?

3) Anything else I might have overlooked?

P.S. the 10gbe switches are connected with 2 cables each to the network backbone switch.

Hi Danny,
To date, 100% of the time this problem has turned out to not be caused by Hyper-V. So, treat this as you would if the exact same thing happened on a physical system. You have a machine, connected to a switch, and the switch is connected to another switch, and the other switch has some path to an Internet-connected firewall. If it’s getting DHCP, then you know for a fact that it has layer two connectivity to whatever distributes IPs. Layer 2 is so simple that it must also have connectivity to everything else in that layer 2 broadcast network. So, now you’re looking for things like bad routing tables and MAC collisions and endpoint firewalls that block ICMP.

great post and I think I understood the most.

But still…

I have a physical machine with one nick behind a firewall which is the DHCP server.
On this physical machine, I have Hyper-V core 2016 running.

I’ve created an external vswitch and assigned the physical nic to it. So the HyperV got a virtual nic attached to the external switch and works just fine.

But when I connect the guest os to the external switch I’ve made, it gets an address thru DHCP but I cannot ping the firewall (Destination host unreachable). I cannot even ping the host machine which should be connected on the same vswitch. Though I can ping from the host to the guest…

When I create an internal switch it all works but I get an IP from the internal switch. I need the machine to have an address out of the range my firewall deals out.

I hope you can help me through this because I’m on the verge of

Many thanks!

Thank you for the information!

The vm will start by sending a SYN package to F5, but the source MAC is replaced by the host when packages is sent on a team member other than the affinitized one.
F5 respond with SYN ACK, but since it contain the wrong MAC, the vm will not receive it and an ACK will not be sent back to F5.
F5 will do a retransmission of the SYN ACK a couple of times and a timeout will occur.

That’s really interesting. Sorry that your problem wasn’t really resolved. What are the symptoms of the problem with the F5 and Dynamic/SI?

F5 BIG-IP has named this feature “auto last hop”. Other LB vendors might have a different name for the same feature. Citrix Netscaler also seems to the same issue.

From https://support.f5.com/csp/article/K13876
“Auto Last Hop is a setting that allows the BIG-IP system to track the source MAC address of incoming connections and return traffic from pools to the source MAC address, regardless of the routing table.”

After weeks with Microsoft premiere support, upgrading firmware/drivers of the nics, testing different teaming and load balancing modes, private hotfixes, Microsoft support concluded that F5 BIG-IP did not support switch independent/dynamic mode or their implementation don’t follow Microsoft “standard”.

We could disable auto last hop on F5 BIG-IP, but since there might be other applications that could face the same issue we decided to change LB mode on the Hyper-V hosts to Hyper-V port. Currently our clusters run either Switch independent/Hyper-V port or LACP/Dynamic.

We had some issues with LACP about 18 months ago. The support case with Microsoft and Cisco solved nothing so we decided to replace our Cisco edge switches with switches from a different vendor. We have not had any issues with LACP since.

You seems to be familiar with this thread regarding the source mac replacement issue, but your view on LACP have changed. 🙂
https://social.technet.microsoft.com/Forums/office/en-US/b698bf90-153b-466b-a078-8355ce730a91/dynamic-lbfo-load-balancing-mode-causing-issues

I was a BIG fan of LACP and I used it on EVERYTHING. My little Netgear switch handled it great, and loved it!
And then one day, someone reported to me that, when one leg of a LACP system failed while connected to a fancy eleventy-billion dollar Cisco switch, it took over a minute for communications to transfer to the other leg. So, we spent hours troubleshooting it with our Cisco engineers. Then we opened up a case with Microsoft that stretched over weeks. Their final conclusion: “Meh, that happens.” Seriously, that was it. I mean, I’m paraphrasing a bit, but they found nothing wrong and took no interest in pursuing it. The more I dug, the more I found out that our experience lies somewhere between semi-rare and common.
So, I did a lot of performance testing. Even under stress, a functioning LACP system doesn’t give much more than Switch Independent. In normal operating conditions, no difference could be detected. Switch Independent has a greater success rate with a nearly identical performance profile. So, I use Switch Independent.

I have not seen a report about the MAC causing issues before. Is it only with 3rd party load balancers or does it manifest in other uses? What mode do you use instead?