Skip to content
Snippets Groups Projects
New look: Off

Black and White sensor lenovo-21bx

    • Open Issue created by Petr Hodina @phodina

    Hi @bodonoghue ,

    you mentioned you have schematics for the B+W sensor for the Windows Hello Camera and also dump of the Windows init stream for the sensor.

    Is there a way to help on bringing up the sensor?

    Designs

      Child items ...

      2. Activity

      • Bryan O'Donoghue
        Bryan O'Donoghue @bodonoghue ·
        Maintainer

        So the main thing to do here is to upstream a single MIPI lane configuration to the B+W - monochrome sensor.

        Intel has done a bunch of work getting sensors upstream to their credit. Intel's typical configuration for monochrome sensors is two data lane mode, whereas qcom does 1 lane.

        In a practical sense its not a big change but, first step here is to locate the relevant init sequence and upstream it + necessary driver description.

        I can probably find the sequence..

      • Petr Hodina
        Petr Hodina @phodina ·
        Author

        Okay, here are some additional information from your valuable research so far :-)

        • The B+W sensor is ov9734. As the datasheet describes it has just single MIPI line.

        • The sensor has XSHUTDN pin that is used for power down and reset (active low with internal pull down resistor)

        • There's additional pin SID that determines the I2C address (SCCB address selection), however based on your research the address seems to be 0x36. There's also a note: SID pin should be pulled low for device address 0x6C and pulled high for device address 0x20.

        • The I2C pins correspond to the signals cci_i2c_sda2/cci_i2c_scl2 on cci1_i2c0

        • The regulators for the sensor are: vreg_l3q, vreg_l5q and vreg_l1q providing dovdd, avdd and dvdd respectively

        • The input clock for the sensor is driven by MCLK4 signal

        • The sensor was upstreamed by Intel and can be found under drivers/media/i2c/ov9734.c

        Based on the commit https://git.codelinaro.org/bryan.odonoghue/kernel/-/commit/6009256d8422fb97c6a063c008e33b657e974b9a

        image

      • Petr Hodina
        Petr Hodina @phodina ·
        Author

        And the datasheet for the sensor: OV9734.pdf

      • Petr Hodina
        Petr Hodina @phodina ·
        Author

        Since the data from the sensor will be RAW it will need processing right> But the software ISP should be enough. There's no need for the Spectra IDSP though offloading to hardware would be preferred.

      • Petr Hodina
        Petr Hodina @phodina ·
        Author

        Would you happen to know which pins are dedicated to the IR LED? Does it use PWM through some dedicated peripheral on the SoC or it's simply ON/OFF for the GPIO?

        Is it also safe to play with the PWM? As eyes are sensitive to light there should be some additional HW that would ensure that the duty cycle of the PWM is within hygienic limits.

        Also as it should be using structured light do we know what kind of pattern mask is on top of the IR LED? It must have been calibrated during manufacturing and the reference image against plain surface stored somewhere.

      • Bryan O'Donoghue
        Bryan O'Donoghue @bodonoghue ·
        Maintainer

        We have a patch for it

        https://github.com/SoMainline/linux/commit/e97c7aa76ce438f2f582784c51b416b1626ca1b1.patch

        I'm not sure those pins are PWMable but I will check.

      • Bryan O'Donoghue
        Bryan O'Donoghue @bodonoghue ·
        Maintainer

        SoftISP will be fine but will require addition of support for the B+W pixel format so actually work todo in that area.

        Edited by Bryan O'Donoghue
      • Petr Hodina
        Petr Hodina @phodina ·
        Author

        FIY there's a Linux solution for facial authentication that could leverage the HW - IR LED and B+W sensor

        https://github.com/boltgolt/howdy

      • Petr Hodina
        Petr Hodina @phodina ·
        Author

        Seems the sensor depends on ACPI. That should be patched out as that's not needed on this ARM laptop and we can directly access the sensor, right?

        At least based on the kconfig:

        Depends on: MEDIA_SUPPORT [=m] && VIDEO_DEV [=m] && VIDEO_CAMERA_SENSOR [=y] && (ACPI [=n] || COMPILE_TEST [=n])
      • Bryan O'Donoghue
        Bryan O'Donoghue @bodonoghue ·
        Maintainer

        Yeah it needs OF binding support added

      • Petr Hodina
        Petr Hodina @phodina ·
        Author

        Any idea how to control the IR LED? Applied the patch, but it's not a GPIO so can't be controlled through libgpiod tools.

        $ tree /sys/firmware/devicetree/base/soc@0/spmi@c440000/pmic@1/led-controller@ee00/led-0/
/sys/firmware/devicetree/base/soc@0/spmi@c440000/pmic@1/led-controller@ee00/led-0/
├── color
├── flash-max-microamp
├── flash-max-timeout-us
├── function
├── function-enumerator
├── led-max-microamp
├── led-sources
└── name
      • Petr Hodina
        Petr Hodina @phodina ·
        Author

        Looks like Johan rewrote the PM8008 PMIC that controls the IR LED

        https://lore.kernel.org/all/20240506150830.23709-8-johan+linaro@kernel.org/T/

        In the meantime I looked at the sensor and made some modifications. Seems the kernel module is loaded, but I can't see the sensors in the userspace:

        $ lsmod | grep ov
ov9734                 16384  0 
ov5675                 16384  0 
v4l2_fwnode            20480  4 ov9734,ov5675,qcom_camss
v4l2_async             20480  4 ov9734,ov5675,qcom_camss,v4l2_fwnode
videodev              221184 10 ov9734,venus_enc,venus_dec,ov5675,qcom_camss,venus_core,v4l2_mem2mem,v4l2_fwnode,videobuf2_v4l2,v4l2_async
mc                     57344  8 ov9734,ov5675,qcom_camss,v4l2_mem2mem,videobuf2_v4l2,videobuf2_common,v4l2_async,videodev
i2c_core               53248 16 i2c_dev,ov9734,ov5675,qcom_pm8008,regmap_i2c,v4l2_async,i2c_hid_of_elan,videodev,i2c_qcom_cci,panel_edp,drm_display_helper,drm_kms_helper,i2c_qcom_geni,i2c_hid_of,i2c_hid,drm

$ cam -l
[0:28:48.585892112] [6912]  INFO Camera camera_manager.cpp:284 libcamera v0.2.0+102-99a77ff7
Available cameras:

        Here's the patch for the changes to the sensor

        0002-drivers-media-i2c-ov9734-Adujst-the-sensor-for-lenov.patch

      • Petr Hodina
        Petr Hodina @phodina ·
        Author

        @bodonoghue if I remove the offending lines for defining the MIPI connection to csiphy the RGB sensor still works. https://git.codelinaro.org/bryan.odonoghue/kernel/-/commit/6009256d8422fb97c6a063c008e33b657e974b9a#1aee5e69b97462728f196577f5af0b26e55e73c8_594_595

        So how exactly are the two cameras connected? Is there some mux that switches which MIPI lane(s) is connected to the csiphy? Or do the sensors have a dedicated port with dedicated lanes for each sensor?

        Also defining the other cci interface for the ov9734 sensor causes these error messages in the kernel log:

        [    2.946498] sc8280xp-tlmm f100000.pinctrl: pin GPIO_123 already requested by 988000.serial; cannot claim for ac4b000.cci
[    2.946501] sc8280xp-tlmm f100000.pinctrl: error -EINVAL: pin-123 (ac4b000.cci)
[    2.946504] sc8280xp-tlmm f100000.pinctrl: error -EINVAL: could not request pin 123 (GPIO_123) from group gpio123  on device f100000.pinctrl
[    2.946506] i2c-qcom-cci ac4b000.cci: Error applying setting, reverse things back
[    2.946700] i2c-qcom-cci ac4c000.cci: Found 19200000 cci clk rate while 37500000 was expected

        Based on the sc8280xp.dtsi the camss has 4 interfaces csiphy{0,1,2,3}.

        Edited by Petr Hodina
      • Petr Hodina
        Petr Hodina @phodina ·
        Author

        @bodonoghue any chance to describe in words the schematics for the Lenovo Laptop?

      • Laurent Pinchart mentioned in commit libcamera@f326eb4b

        mentioned in commit libcamera@f326eb4b

      Please register or sign in to reply