Symptoms

The device is not found by an any Impact Acquire based application and the kernel log files indicate memory problems in the kernel module when initializing the device or reserving memory for DMA.

Cause

The Linux kernel module for BVS CA-BN devices is designed by default to use coherent memory and an IOMMU for DMA. System memory on a host is described as 'coherent' when the hardware ensures that CPU caches are flushed or invalidated automatically for this memory area. On host systems that use the x86_64 architecture all memory is usually coherent but on host systems using an ARM CPU there may be limited amounts of coherent memory, or indeed, no coherent memory and no IOMMU at all.

Resolution

Note: Explicitly flushing or invalidating the CPU cache is an operation that can tie up CPU resources and is likely to lead to higher CPU usage. For this reason we do not recommend using a host system without IOMMU and coherent memory. Please check the documentation from the hardware manufacturer and, if possible, always turn on any options to use these features. This is often done using BIOS options on a PC or a device tree on an ARM-based system.

If it is not possible to use coherent memory on your system or the coherent memory available appears to be too small (see kernel logs) it should nevertheless still be possible to use the BVS CA-BN device by loading the kernel module using an extra parameter that explicitly specifies NOT to use coherent memory. In this case normal memory allocations will be made and the cache will be explicitly flushed or invalidated, as appropriate.

The kernel parameter to use is no_coherent_mem=1. There are several ways to specify it.

It can be specified manually on the command line with the "modprobe" command if you are loading the kernel module manually:

modprobe mvpci no_coherent_mem=1

You may add a new line to the file "/etc/modules" or create a new file e.g. "/etc/modules-load.d/mvpci.conf". The file used must contain a line like this:

mvpci no_coherent_mem=1