PCIPopulationRules

Some systems have PCI population rules that must be followed to avoid PCI errors.

For example the Intel SR2400 comprised of an Intel Server Board SE7520JR2 combined with the Passive Full Height PCI-X Riser Card required add-on cards to be populated starting with the furthest slots from the mainboard connector. When cards such as NICs are inserted in the middle or lower slots, newer BIOS versions halt during POST and inform the user about the population rules. Older BIOS versions would not catch this and users would intermittently experience PCI errors. With certain BIOS settings and OS versions, these would appear as NMIs.

Any large passive riser should be suspect if you are getting PCI errors. It is possible that signal reflections are occurring. If no population rules are available for your hardware, just assume you should move cards to the end of the PCI bus. This both dampens any reflections, and avoids placing cards where reflections would tend to cancel out the signals.

Another possibility with riser cards is that the retention bracket is exerting sideways force on the PCI connectors. This was the case with certain steel brackets included in some Supermicro 1RU server chassis. The brackets were slightly curved from the process of rolling the metal edges. This exerted a slight sideways force on the PCI connector and was suspected of causing PCI errors in a large fraction of servers with Myrinet cards installed. The cards were originally suspected but eventually it was discovered that removing the metal bracket reliably stopped the errors.