Write problem with miniPCIe FD on Linux

[olbender]

Hi,

I have some strange problems with miniPCIe (PCAN-miniPCIe FD) on Linux and SocketCAN. I use the card with four channels in an PCEngines APU (apu4: https://pcengines.ch/apu4d4.htm). I have tested it with three different CAN devices, and I see the same problem on all. I can read without problems, but when writing I get "No buffer space available" (err=105). After the error is triggered the interface freezes completely and reading is also blocked.

The interesting thing is that when I use PCAN-USB interfaces instead (drop-in replacement) it works perfectly (same computer, same OS, same software), even when running two such interfaces simultaneously. I also tried with two different new APU units, as well as two different PCAN-miniPCIe FD devices, with the same results. Therefore I think it might relate to either the miniPCIe hardware implementation on the APU, or the Linux peak_pciefd (while the peak_usb works fine). Or might it be something related to the "FD", as the PCAN-USB did not have any flexible data? Could I get some help to debug this?

I saw some suggestions for kernel parameters, but I didn't test anything yet as I would first want to understand what is going on.

Perhaps this is related? viewtopic.php?f=144&t=3354&p=10055&hili ... upt#p10055

Code: Select all

$ uname -a
Linux seacat-0 6.0.11-arch1-1 #1 SMP PREEMPT_DYNAMIC Fri, 02 Dec 2022 17:25:31 +0000 x86_64 GNU/Linux

Code: Select all

# dmesg | grep pci
[    6.199217] peak_pciefd 0000:05:00.0: 4x CAN-FD PCAN-PCIe FPGA v3.2.1:
[    6.200628] peak_pciefd 0000:05:00.0: can0 at reg_base=0x000000006f2e9645 irq=64
[    6.201440] peak_pciefd 0000:05:00.0: can1 at reg_base=0x00000000ac7722b2 irq=64
[    6.205225] peak_pciefd 0000:05:00.0: can2 at reg_base=0x0000000032cddb9f irq=64
[    6.206277] peak_pciefd 0000:05:00.0: can3 at reg_base=0x0000000028aac725 irq=64

Code: Select all

# cat /proc/interrupts 
            CPU0       CPU1       CPU2       CPU3       
   0:         51          0          0          0   IO-APIC    2-edge      timer
   4:          0          0          0         18   IO-APIC    4-edge      ttyS0
   7:          0          0          0          0   IO-APIC    7-fasteoi   pinctrl_amd
   8:          0          0          1          0   IO-APIC    8-edge      rtc0
   9:          0          0          0          0   IO-APIC    9-fasteoi   acpi
  16:          0          0          0          0   IO-APIC   16-fasteoi   mmc0
  18:          0    3504036          0          0   IO-APIC   18-fasteoi   ehci_hcd:usb1, ehci_hcd:usb2
  19:      76599          0          0          0   IO-APIC   19-fasteoi   ahci[0000:00:11.0]
  25:          0          0          0          0   PCI-MSI 34816-edge      PCIe PME, aerdrv
  27:          0          0          0          0   PCI-MSI 36864-edge      PCIe PME, aerdrv
  29:          0          0          0          0   PCI-MSI 38912-edge      PCIe PME, aerdrv
  31:          0          0          0          0   PCI-MSI 40960-edge      PCIe PME, aerdrv
  32:          0          0          0          0   PCI-MSI 43008-edge      PCIe PME, aerdrv
  33:          0          0          0          0  amd_gpio   89  switch1
  34:   49755036          0          0          0   PCI-MSI 262144-edge      xhci_hcd
  35:          0          0          0          0   PCI-MSI 262145-edge      xhci_hcd
  36:          0          0          0          0   PCI-MSI 262146-edge      xhci_hcd
  37:          0          0          0          0   PCI-MSI 262147-edge      xhci_hcd
  38:          0          0          0          0   PCI-MSI 262148-edge      xhci_hcd
  39:          0          1          0          0   PCI-MSI 524288-edge      enp1s0
  40:          0          0     529638          0   PCI-MSI 524289-edge      enp1s0-rx-0
  41:          0          0          0     112652   PCI-MSI 524290-edge      enp1s0-rx-1
  42:     127204          0          0          0   PCI-MSI 524291-edge      enp1s0-tx-0
  43:          0      48789          0          0   PCI-MSI 524292-edge      enp1s0-tx-1
  45:          3          0          0          0   PCI-MSI 1048576-edge      enp2s0
  46:          0     385302          0          0   PCI-MSI 1048577-edge      enp2s0-rx-0
  47:          0          0      47336          0   PCI-MSI 1048578-edge      enp2s0-rx-1
  48:          0          0          0      46306   PCI-MSI 1048579-edge      enp2s0-tx-0
  49:     100927          0          0          0   PCI-MSI 1048580-edge      enp2s0-tx-1
  57:          0          0          0          0   PCI-MSI 131073-edge      ccp-1
  59:          0          0          0          0   PCI-MSI 2097152-edge      enp4s0
  60:      45273          0          0          0   PCI-MSI 2097153-edge      enp4s0-rx-0
  61:          0      45273          0          0   PCI-MSI 2097154-edge      enp4s0-rx-1
  62:          0          0      45273          0   PCI-MSI 2097155-edge      enp4s0-tx-0
  63:          0          0          0      45273   PCI-MSI 2097156-edge      enp4s0-tx-1
  64:          0          0    1293332          0   IO-APIC   16-fasteoi   peak_pciefd, peak_pciefd
 NMI:        490        266        262        258   Non-maskable interrupts
 LOC:   19473766   15957702   14736671   14823410   Local timer interrupts
 SPU:          0          0          0          0   Spurious interrupts
 PMI:        490        266        262        258   Performance monitoring interrupts
 IWI:          2          3          0          3   IRQ work interrupts
 RTR:          0          0          0          0   APIC ICR read retries
 RES:     860307     875618     862165     846639   Rescheduling interrupts
 CAL:    6078259    5873478    5880196    5806704   Function call interrupts
 TLB:       7020       6739       6216       6168   TLB shootdowns
 TRM:          0          0          0          0   Thermal event interrupts
 THR:          0          0          0          0   Threshold APIC interrupts
 DFR:          0          0          0          0   Deferred Error APIC interrupts
 MCE:          0          0          0          0   Machine check exceptions
 MCP:        291        291        291        291   Machine check polls
 ERR:          1
 MIS:          0
 PIN:          0          0          0          0   Posted-interrupt notification event
 NPI:          0          0          0          0   Nested posted-interrupt event
 PIW:          0          0          0          0   Posted-interrupt wakeup event

I start the device with:

Code: Select all

ip link set can0 up type can bitrate 250000

Thank you!

[M.Maidhof]

Hi,

this could be an IRQ problem on that system, which could be fixed by switching to MSI instead of using INT A for the interrupt management. To use MSI for our PCAN miniPCIe FD cards, you have to install our latest peak-linux-driver in SocketCAN mode and edit the pcan.conf to enable MSI:

Download the driver: https://www.peak-system.com/quick/PCAN-Linux-Driver

Untar the package:

tar -xzf peak-linux-driver-8.15.2.tar.gz
cd peak-linux-driver-8.15.2
make clean
make NET=NETDEV_SUPPORT
sudo make install
sudo modprobe pcan

please edit /etc/modprobe.d/pcan.conf and add the red line, to enable MSI support:

# pcan - automatic made entry, begin --------
# if required add options and remove comment
# options pcan type=isa,sp
options pcan fdusemsi=1
install pcan modprobe --ignore-install pcan
# pcan - automatic made entry, end ----------

Reboot your system once to load the driver with that new option.

check that cat /proc/pcan will show differnt IRQs for the card channels and make your test again.

regards

Michael