Skip to content
Snippets Groups Projects
New look: Off

Issue with SIM7600 on Raspberry Pi after EMI: Device switching from qmi_wwan to simtech plugin

    • Open Issue created by Zsolt Szabó @szabozsolt.sfc

    Setup Details

    I am using a Raspberry Pi as a vehicle tracking device, where mobile internet connectivity is provided via a SIM7600 modem using qmi_wwan and NetworkManager.

    My current NetworkManager configuration:

    [connection]
id=SfcMobileConnection
uuid=fc5d1c10-693c-4399-94c3-0de518f70083
type=gsm
autoconnect-priority=999
autoconnect-retries=0
interface-name=cdc-wdm0

[gsm]
apn=internet.vodafone.net
#apn=telematics.net

[ipv4]
method=auto

[ipv6]
addr-gen-mode=default
method=disabled

[proxy]

    Normal Operation

    Under normal operation, mmcli -m sim7600 returns the following:

    sfc@raspberrypi:~ $ mmcli -m sim7600
  -----------------------------------
  General  |                    path: /org/freedesktop/ModemManager1/Modem/0
           |               device id: 4b4ec4b1d39081f2cc4ceefc5c627e3a6e4ef93c
  -----------------------------------
  Hardware |            manufacturer: QUALCOMM INCORPORATED
           |                   model: 0
           |       firmware revision: LE11B12SIM7600M22
           |          carrier config: ROW_Gen_VoLTE
           | carrier config revision: 05010813
           |            h/w revision: 10000
           |               supported: gsm-umts, lte
           |                 current: gsm-umts, lte
           |            equipment id: 867584032463817
  -----------------------------------
  System   |                  device: sim7600
           |                 drivers: qmi_wwan, option
           |                  plugin: simtech
           |            primary port: cdc-wdm0
           |                   ports: cdc-wdm0 (qmi), ttyUSB2 (ignored), ttyUSB3 (gps),
           |                          ttyUSB4 (at), ttyUSB5 (at), ttyUSB6 (audio), wwan0 (net)
  -----------------------------------
  Status   |                    lock: sim-pin2
           |          unlock retries: sim-pin (3), sim-puk (10), sim-pin2 (3), sim-puk2 (10)
           |                   state: connected
           |             power state: on
           |             access tech: lte
           |          signal quality: 41% (recent)
  -----------------------------------
  Modes    |               supported: allowed: 2g; preferred: none
           |                          allowed: 3g; preferred: none
           |                          allowed: 4g; preferred: none
           |                          allowed: 2g, 3g; preferred: 3g
           |                          allowed: 2g, 3g; preferred: 2g
           |                          allowed: 2g, 4g; preferred: 4g
           |                          allowed: 2g, 4g; preferred: 2g
           |                          allowed: 3g, 4g; preferred: 4g
           |                          allowed: 3g, 4g; preferred: 3g
           |                          allowed: 2g, 3g, 4g; preferred: 4g
           |                          allowed: 2g, 3g, 4g; preferred: 3g
           |                          allowed: 2g, 3g, 4g; preferred: 2g
           |                 current: allowed: 2g, 3g, 4g; preferred: 4g
  -----------------------------------
  Bands    |               supported: egsm, dcs, utran-1, utran-5, utran-8, eutran-1, eutran-3,
           |                          eutran-5, eutran-7, eutran-8, eutran-20, eutran-38, eutran-40,
           |                          eutran-41
           |                 current: egsm, dcs, utran-1, utran-5, utran-8, eutran-1, eutran-3,
           |                          eutran-5, eutran-7, eutran-8, eutran-20, eutran-38, eutran-40,
           |                          eutran-41
  -----------------------------------
  IP       |               supported: ipv4, ipv6, ipv4v6
  -----------------------------------
  3GPP     |                    imei: 867584032463817
           |           enabled locks: fixed-dialing
           |             operator id: 21670
           |           operator name: vodafone HU
           |            registration: home
           |    packet service state: attached
  -----------------------------------
  3GPP EPS |    ue mode of operation: ps-2
           |     initial bearer path: /org/freedesktop/ModemManager1/Bearer/0
           |  initial bearer ip type: ipv4
  -----------------------------------
  SIM      |        primary sim path: /org/freedesktop/ModemManager1/SIM/0
           |          sim slot paths: slot 1: /org/freedesktop/ModemManager1/SIM/0 (active)
           |                          slot 2: none
  -----------------------------------
  Bearer   |                   paths: /org/freedesktop/ModemManager1/Bearer/1

    Also, mmcli -M lists:

    sfc@raspberrypi:~ $ mmcli -M
    /org/freedesktop/ModemManager1/Modem/0 [QUALCOMM INCORPORATED] 0

    Problem

    Occasionally, an EMI event disrupts the system. Based on logs, the kernel detects an issue with the USB device:

    Jan 17 18:21:11 raspberrypi kernel: usb 1-1-port2: disabled by hub (EMI?), re-enabling...
Jan 17 18:21:12 raspberrypi kernel: usb 1-1.2: USB disconnect, device number 17
Jan 17 18:21:12 raspberrypi kernel: usb 1-1.2.1: USB disconnect, device number 18
Jan 17 18:21:12 raspberrypi kernel: ftdi_sio ttyUSB0: FTDI USB Serial Device converter now disconnected from ttyUSB0
Jan 17 18:21:12 raspberrypi kernel: ftdi_sio 1-1.2.1:1.0: device disconnected
Jan 17 18:21:12 raspberrypi kernel: usb 1-1.2.2: USB disconnect, device number 19
Jan 17 18:21:12 raspberrypi kernel: ftdi_sio ttyUSB1: FTDI USB Serial Device converter now disconnected from ttyUSB1
Jan 17 18:21:12 raspberrypi kernel: ftdi_sio 1-1.2.2:1.0: device disconnected
Jan 17 18:21:12 raspberrypi kernel: usb 1-1.2: new high-speed USB device number 20 using dwc_otg
Jan 17 18:21:12 raspberrypi kernel: usb 1-1.2: New USB device found, idVendor=1a40, idProduct=0101, bcdDevice= 1.11
Jan 17 18:21:12 raspberrypi kernel: usb 1-1.2: New USB device strings: Mfr=0, Product=1, SerialNumber=0
Jan 17 18:21:12 raspberrypi kernel: usb 1-1.2: Product: USB 2.0 Hub
Jan 17 18:21:12 raspberrypi kernel: hub 1-1.2:1.0: USB hub found
Jan 17 18:21:12 raspberrypi kernel: hub 1-1.2:1.0: 4 ports detected
Jan 17 18:21:12 raspberrypi kernel: usb 1-1.2.1: new full-speed USB device number 21 using dwc_otg
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.1: New USB device found, idVendor=0403, idProduct=6001, bcdDevice= 6.00
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.1: Product: FT232R USB UART
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.1: Manufacturer: FTDI
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.1: SerialNumber: B004A1W1
Jan 17 18:21:13 raspberrypi kernel: ftdi_sio 1-1.2.1:1.0: FTDI USB Serial Device converter detected
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.1: Detected FT232R
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.1: FTDI USB Serial Device converter now attached to ttyUSB0
Jan 17 18:21:13 raspberrypi mtp-probe[21007]: checking bus 1, device 21: "/sys/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.2/1-1.2.1"
Jan 17 18:21:13 raspberrypi mtp-probe[21007]: bus: 1, device: 21 was not an MTP device
Jan 17 18:21:13 raspberrypi mtp-probe[21024]: checking bus 1, device 21: "/sys/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.2/1-1.2.1"
Jan 17 18:21:13 raspberrypi mtp-probe[21024]: bus: 1, device: 21 was not an MTP device
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.2: new full-speed USB device number 22 using dwc_otg
Jan 17 18:21:13 raspberrypi ModemManager[465]: <warn>  [modem0] port cdc-wdm0 timed out 4 consecutive times
Jan 17 18:21:13 raspberrypi ModemManager[465]: <warn>  [modem0/bearer1] reloading stats failed: QMI operation failed: Transaction timed out
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.2: New USB device found, idVendor=0403, idProduct=6001, bcdDevice= 6.00
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.2: Product: FT232R USB UART
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.2: Manufacturer: FTDI
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.2: SerialNumber: B004A1WE
Jan 17 18:21:13 raspberrypi kernel: ftdi_sio 1-1.2.2:1.0: FTDI USB Serial Device converter detected
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.2: Detected FT232R
Jan 17 18:21:13 raspberrypi kernel: usb 1-1.2.2: FTDI USB Serial Device converter now attached to ttyUSB1
Jan 17 18:21:13 raspberrypi mtp-probe[21025]: checking bus 1, device 22: "/sys/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.2/1-1.2.2"
Jan 17 18:21:13 raspberrypi mtp-probe[21025]: bus: 1, device: 22 was not an MTP device
Jan 17 18:21:13 raspberrypi mtp-probe[21042]: checking bus 1, device 22: "/sys/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.2/1-1.2.2"
Jan 17 18:21:13 raspberrypi mtp-probe[21042]: bus: 1, device: 22 was not an MTP device
Jan 17 18:21:15 raspberrypi ModemManager[465]: <info>  [base-manager] couldn't check support for device '/sys/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.2/1-1.2.1': not supported by any plugin
Jan 17 18:21:16 raspberrypi ModemManager[465]: <info>  [base-manager] couldn't check support for device '/sys/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.2/1-1.2.2': not supported by any plugin
Jan 17 18:21:40 raspberrypi ModemManager[465]: <warn>  [modem0] port cdc-wdm0 timed out 5 consecutive times
Jan 17 18:21:43 raspberrypi ModemManager[465]: <warn>  [modem0] port cdc-wdm0 timed out 6 consecutive times
Jan 17 18:21:43 raspberrypi ModemManager[465]: <warn>  [modem0/bearer1] reloading stats failed: QMI operation failed: Transaction timed out
Jan 17 18:21:43 raspberrypi kernel: qmi_wwan 1-1.4:1.5: Unexpected error -71
Jan 17 18:21:43 raspberrypi ModemManager[465]: <warn>  Cannot read from istream: connection broken
Jan 17 18:21:43 raspberrypi ModemManager[465]: <info>  [modem0] port 'cdc-wdm0' no longer controllable, reprobing
Jan 17 18:21:43 raspberrypi kernel: ------------[ cut here ]------------
Jan 17 18:21:43 raspberrypi kernel: WARNING: CPU: 2 PID: 642 at drivers/usb/core/urb.c:379 usb_submit_urb+0x530/0x53c
Jan 17 18:21:43 raspberrypi kernel: URB a179cd32 submitted while active
Jan 17 18:21:43 raspberrypi kernel: Modules linked in: qmi_wwan cdc_wdm option usb_wwan cmac algif_hash aes_arm_bs crypto_simd cryptd algif_skcipher af_alg bnep brcmfmac_wcc vc4 brcmfmac snd_soc_hdmi_codec drm_display_helper cec drm_dma_helper drm_kms_helper binfmt_misc snd_soc_core brcmutil ftdi_sio usbserial cfg80211 bcm2835_codec(C) raspberrypi_hwmon snd_compress hci_uart snd_pcm_dmaengine btbcm v4l2_mem2mem bluetooth bcm2835_v4l2(C) bcm2835_isp(C) bcm2835_mmal_vchiq(C) videobuf2_vmalloc snd_bcm2835(C) videobuf2_dma_contig videobuf2_memops snd_pcm videobuf2_v4l2 videodev ecdh_generic ecc rfkill snd_timer vc_sm_cma(C) snd videobuf2_common mc raspberrypi_gpiomem uio_pdrv_genirq uio drm fuse dm_mod drm_panel_orientation_quirks backlight ip_tables x_tables ipv6 i2c_bcm2835 fixed
Jan 17 18:21:43 raspberrypi kernel: CPU: 2 PID: 642 Comm: qmi-proxy Tainted: G         C         6.6.51+rpt-rpi-v7 #1  Raspbian 1:6.6.51-1+rpt3
Jan 17 18:21:43 raspberrypi kernel: Hardware name: BCM2835
Jan 17 18:21:43 raspberrypi kernel:  unwind_backtrace from show_stack+0x18/0x1c
Jan 17 18:21:43 raspberrypi kernel:  show_stack from dump_stack_lvl+0x50/0x68
Jan 17 18:21:43 raspberrypi kernel:  dump_stack_lvl from __warn+0x80/0x11c
Jan 17 18:21:43 raspberrypi kernel:  __warn from warn_slowpath_fmt+0x12c/0x198
Jan 17 18:21:43 raspberrypi kernel:  warn_slowpath_fmt from usb_submit_urb+0x530/0x53c
Jan 17 18:21:43 raspberrypi kernel:  usb_submit_urb from service_outstanding_interrupt+0x7c/0xf0 [cdc_wdm]
Jan 17 18:21:43 raspberrypi kernel:  service_outstanding_interrupt [cdc_wdm] from wdm_read+0x2ec/0x398 [cdc_wdm]
Jan 17 18:21:43 raspberrypi kernel:  wdm_read [cdc_wdm] from vfs_read+0x98/0x2dc
Jan 17 18:21:43 raspberrypi kernel:  vfs_read from ksys_read+0x78/0xf8
Jan 17 18:21:43 raspberrypi kernel:  ksys_read from __sys_trace_return+0x0/0x10
Jan 17 18:21:43 raspberrypi kernel: Exception stack(0xbbaf9fa8 to 0xbbaf9ff0)
Jan 17 18:21:43 raspberrypi kernel: 9fa0:                   00000800 7eecf39c 00000007 7eecf39c 00000800 00000000
Jan 17 18:21:43 raspberrypi kernel: 9fc0: 00000800 7eecf39c 00000007 00000003 7eecf398 01eaeeb0 00000001 76f37450
Jan 17 18:21:43 raspberrypi kernel: 9fe0: 76f36fc0 7eecf300 00000000 76a0d6ac
Jan 17 18:21:43 raspberrypi kernel: ---[ end trace 0000000000000000 ]---
Jan 17 18:21:43 raspberrypi kernel: qmi_wwan 1-1.4:1.5: usb_submit_urb failed with result -16
Jan 17 18:21:43 raspberrypi kernel: 
Jan 17 18:21:43 raspberrypi kernel: WARN::dwc_otg_hcd_urb_dequeue:639: Timed out waiting for FSM NP transfer to complete on 6
Jan 17 18:21:43 raspberrypi kernel: 
Jan 17 18:21:43 raspberrypi kernel: WARN::dwc_otg_hcd_urb_dequeue:639: Timed out waiting for FSM NP transfer to complete on 3
Jan 17 18:21:43 raspberrypi kernel: 
Jan 17 18:21:43 raspberrypi kernel: WARN::dwc_otg_hcd_urb_dequeue:639: Timed out waiting for FSM NP transfer to complete on 2
Jan 17 18:21:43 raspberrypi NetworkManager[454]: <info>  [1737134503.7848] device (cdc-wdm0): state change: activated -> unmanaged (reason 'removed', sys-iface-state: 'removed')
Jan 17 18:21:44 raspberrypi NetworkManager[454]: <info>  [1737134504.2956] manager: NetworkManager state is now DISCONNECTED
Jan 17 18:21:44 raspberrypi dbus-daemon[397]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service' requested by ':1.7' (uid=0 pid=454 comm="/usr/sbin/NetworkManager --no-daemon")
Jan 17 18:21:44 raspberrypi NetworkManager[454]: <warn>  [1737134504.3068] modem-broadband[cdc-wdm0]: failed to disconnect modem: GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: Object does not exist at path “/org/freedesktop/ModemManager1/Modem/0”
Jan 17 18:21:44 raspberrypi systemd[1]: Starting NetworkManager-dispatcher.service - Network Manager Script Dispatcher Service...
Jan 17 18:21:44 raspberrypi dbus-daemon[397]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
Jan 17 18:21:44 raspberrypi systemd[1]: Started NetworkManager-dispatcher.service - Network Manager Script Dispatcher Service.
...
...
...
Jan 17 18:21:46 raspberrypi ModemManager[465]: <info>  [device sim7600] creating modem with plugin 'simtech' and '7' ports
...
...
...
Jan 17 18:21:49 raspberrypi ModemManager[465]: <info>  [base-manager] couldn't check support for device '/sys/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.2/1-1.2.1': not supported by any plugin
Jan 17 18:21:50 raspberrypi ModemManager[465]: <info>  [base-manager] couldn't check support for device '/sys/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.2/1-1.2.2': not supported by any plugin
...
...
...
Jan 17 18:23:16 raspberrypi ModemManager[465]: <warn>  [modem1] couldn't start initialization: Transaction timed out
Jan 17 18:23:16 raspberrypi ModemManager[465]: <error> [modem1] fatal error initializing: Fatal error: modem is unusabl

    At one point in my software, it checks the following:

    if os.path.exists('/dev/cdc-wdm0'):
    logger.info('SIM7600 modem device is available')

    Which is equivalent to a "ls /dev/cdc-wdm*" command, and it does not indicate that the cdc_wdm0 device is missing. So after the EMI error I guess this device is present, because I get True back. But after that I can no longer connect to the mobile network:

    Jan 17 18:26:54 raspberrypi NetworkManager[454]: <info>  [1737134814.8471] agent-manager: agent[034b44c83a03525b,:1.3608/nmcli-connect/1000]: agent registered
Jan 17 18:26:54 raspberrypi NetworkManager[454]: <info>  [1737134814.8514] audit: op="connection-activate" uuid="fc5d1c10-693c-4399-94c3-0de518f70083" name="SfcMobileConnection" result="fail" reason="No suitable device found for this connection (device eth0 not available because profile is not compatible with device (mismatching interface name))."

    Testing EMI Reproduction

    Unfortunately, I cannot reproduce the EMI error under development conditions, so I am trying to reproduce the error with the following solution, Maybe not a good way...

    I manually run:

    sudo rmmod qmi_wwan
sudo rmmod cdc_wdm
sudo modprobe qmi_wwan
sudo modprobe cdc_wdm

    Following this, the modem becomes accessible again, but mmcli -m sim7600 now shows:

    sfc@raspberrypi:~ $ sudo mmcli -m sim7600
  --------------------------------
  General  |                 path: /org/freedesktop/ModemManager1/Modem/1
           |            device id: f7239e08263e43b6d8ccab7967e98cd5e2064df6
  --------------------------------
  Hardware |         manufacturer: SIMCOM INCORPORATED
           |                model: SIMCOM_SIM7600E-H
           |    firmware revision: LE11B12SIM7600M22
           |            supported: gsm-umts, lte
           |              current: gsm-umts, lte
           |         equipment id: 867584032463817
  --------------------------------
  System   |               device: sim7600
           |              drivers: qmi_wwan, option
           |               plugin: simtech
           |         primary port: ttyUSB4
           |                ports: ttyUSB2 (ignored), ttyUSB3 (gps), ttyUSB4 (at),
           |                       ttyUSB5 (at), ttyUSB6 (audio)
  --------------------------------
  Status   |       unlock retries: sim-pin (3), sim-puk (10), sim-pin2 (3), sim-puk2 (10)
           |                state: registered
           |          power state: on
           |          access tech: lte
           |       signal quality: 41% (recent)
  --------------------------------
  Modes    |            supported: allowed: 2g; preferred: none
           |                       allowed: 3g; preferred: none
           |                       allowed: 2g, 3g; preferred: none
           |                       allowed: 2g, 3g; preferred: 2g
           |                       allowed: 2g, 3g; preferred: 3g
           |              current: allowed: any; preferred: none
  --------------------------------
  IP       |            supported: ipv4, ipv6, ipv4v6
  --------------------------------
  3GPP     |                 imei: 867584032463817
           |          operator id: 21670
           |        operator name: One HU
           |         registration: home
           | packet service state: attached
  --------------------------------
  3GPP EPS | ue mode of operation: ps-2
  --------------------------------
  SIM      |     primary sim path: /org/freedesktop/ModemManager1/SIM/1
    (-) /org/freedesktop/ModemManager1/Modem/0 [QUALCOMM INCORPORATED] 0
(+) /org/freedesktop/ModemManager1/Modem/1 [SIMCOM INCORPORATED] SIMCOM_SIM7600E-H

    The differences are:

    Befor After
    Primary port: cdc-wdm0 Primary port: ttyUSB4
    cdc-wdm0 (qmi), ttyUSB2 (ignored), ttyUSB3 (gps), | ttyUSB4 (at), ttyUSB5 (at), ttyUSB6 (audio), wwan0 (net) ttyUSB2 (ignored), ttyUSB3 (gps), ttyUSB4 (at), | ttyUSB5 (at), ttyUSB6 (audio)
    Plugin: generic Plugin: simtech

    I tryed this:

    sfc@raspberrypi:~ $ nmcli dev set cdc-wdm0 managed yes
Error: Device 'cdc-wdm0' not found.
sfc@raspberrypi:~ $ ls /dev/cdc-wdm0
/dev/cdc-wdm0

    Analysis

    I suspect that after an EMI event, the ModemManager no longer attaches to cdc-wdm0 via the qmi_wwan driver but instead switches to ttyUSB4 with the simtech plugin. This leads to the NetworkManager failing to establish the connection.

    Questions

    1. How can I ensure the ModemManager consistently uses the generic plugin with qmi_wwan after the device is re-initialized?
    2. Is there a way to automatically rebind cdc-wdm0 to qmi_wwan after a USB reconnect event?
    3. Are there additional configurations I can use in the NetworkManager or udev rules to handle this gracefully?

    Any suggestions or guidance on how to approach this would be greatly appreciated! Thank you! :blush:

    To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information

    Child items ...

    • Activity

    • Dan Williams added Needs Information label

      added Needs Information label

    • Dan Williams
      Dan Williams @dcbw ·
      Owner

      @szabozsolt.sfc What kernel version do you have?

      You likely have the high-level problem correct; after the modem hiccups and you get the kernel warning from cdc-wdm, the driver is wedged and may not be able to enumerate. It also may be the case that the modem itself isn't in a state to expose certain ports.

      One thought: are you able to use uhubctl to power cycle the modem? Most (all?) recent Raspberry Pi devices are supposed to have per-port power control on their onboard USB hubs.

    • Dmitry Sharshakov
      Dmitry Sharshakov @dsseng ·

      Also, is the QMI device still accessible via qmicli after the event? My SIM7600 modem (PID mode 9001) has simtech plugin in use despite no EMI events and overall stable operation.

    Please register or sign in to reply