Accelerating your

intelligent in-cabin
camera design

Cliff CHENG – Leopard Imaging

Chris VARLAMOS – STMicroelectronics
 Agenda

1 In-cabin sensing introduction 4 Design considerations

Leopard Imaging modules

2 ST in-cabin imagers offering 5
based on ST image sensors

Leopard Imaging modules

3 6 Takeaways
offering

2
 Some figures

2M car accident a year

in the U.S.
1M car thefts in 2022 in
the U.S.

deadly car crash in every time a theft

43,000 2022 in the U.S. 30 sec happens in the U.S.

cause of death in children total car thefts value

#1 under the age of 15 $8B in the U.S.

Sources: CDCP, NHTSA, NICB 3

 The story of automobile safety
Fatalities per
100 million miles
1
2
5

Year Safety measure 4

3
1 1968 2-point seatbelt

2 1969 Front headrest 4

3 1973 3-point seatbelt 3
11
4 1979 NHTSA crash testing

5 1985 3rd brake light 5

6 1995 5-star safety rating

2
7 1998 Dual front airbags
6
8 2008 Tire pressure monitoring 7

9 2012 ABS / ESC 8

9
1 9
10 2018 Backup cameras

11 202x In-cabin monitoring

1960 1970 1980 1990 2000 2010 2020 2030

4
Sources: TitleMax, NHTSA, CNBC
 Euro NCAP roadmap for in-cabin monitoring

2023 2025
First camera-based DMS Only direct sensing systems,
assessment from Euro NCAP capable of effectively detecting a
Indirect and direct sensing solutions living being, will be awarded
for Child Presence Detection will
score points

2024 2026
DMS to be mandatory across all DMS to be mandatory in all new
new vehicle platforms vehicles

5
 In-cabin market potential

Highest
growth rate

Source: Yole Intelligence, Imaging for Automotive – June 2023 6

 From Safety Regulations to an
Upgraded User Experience
SAFETY USER EXPERIENCE

• Warning the drivers when distracted, drowsy or
otherwise not paying attention to the road
• Enhancing safety by providing information to the
vehicle’s systems about the status of the occupants
• Authenticating users to access the vehicle
• Monitoring drivers and passengers’ health with a multi-
sensor approach
• Enabling interaction between passenger and large
displays from different position, allowing smart and
intuitive lighting at specific area with gestures control
• Ease access to personal data, cloud services and
secured payments with face authentication
• Enabling videoconferencing, objects left behind, etc.

7
 In-Cabin Camera Evolution

From single DMS system to hybrid in-cabin applications

Separate DMS and OMS Single hybrid DMS and OMS

1 Single DMS camera 2 cameras 3 camera

8
 In-Cabin Sensing Applications

DRIVER MONITORING SYSTEM (DMS) OCCUPANT MONITORING SYSTEM (OMS)

Monitor drivers’ alertness to boost roads safety Monitor passengers’ presence and objects

• Drowsiness • Child Presence Detection (CPD)

• Awareness • Vital Sign Monitoring (VSM)
• Driving Under Influence (DUI) • Smart airbag deployment
• Vital Sign Monitoring (VSM) • Intruder detection
• Hands-off the steering wheel detection • Objects Left-behind
• Head-up displays for Augmented reality (AR) • Video Conferencing
applications • Gesture recognition

Beyond DMS, expanding to new applications: CPD is one main application mandated for North
Vital Monitoring, hands-off and AR head-up displays America and Europe – 5-stars rating on 2023 Euro-NCAP

9

Basic DMS
VB56G4A – 1.5MP mono CFA
• First automotive 3D stack BSI
• Highest sensitivity at 940nm
• Embedded auto-exposure
• Smallest size and very low-power
• Configurable strobe signal
enabling PWM
ST Automotive Image Sensor Portfolio
Advanced DMS
VG5761 – 2.3MP mono CFA
• First global shutter with HDR
• Higher MTF and resolution
• Background subtraction mode
• Context switching
Hybrid DMS/OMS
VB1940 – 5.1MP RGB-IR CFA
• World’s first global and rolling
shutter image sensor
• On-chip processing
• Cybersecurity and ASIL-B
• Highest resolution
• High QE
• Dual exposure
• Context switching
10
 VB56G4A

1.5 MP cost-effective and low-power global shutter image sensor

Parameter Specification Features Applications

Pixel 2.61 um • High 940nm sensitivity while
preserving GSE and MTF • Standalone DMS
Technology 3D stack GS BSI
• Auto Exposure • Gaze tracking for HUD* and
Array Size 1124 x 1364 Augmented Reality overlays
• Micro-lens shift for 30 CRA
Frame Rate 90 fps full resolution

HDR 60 dB

940 QE 25% Benefits

940 MTF High • Optimized pixel to reduce system
cost and improve performance
CFA Options Mono
• Reduce NIR LED power
Package [mm] 6.2 x 6.9 OBGA
• Enables low Z-height

11
 VG5761

2.3 MP HDR with advanced features global shutter image sensor

Parameter Specification Features Applications

Pixel 3.2 um • Industry 1st GS-HDR • Advanced DMS
Technology GS-HDR FSI
• Ambient image subtraction
• Standalone OMS with HDR
Array Size 1944 x 1204
• High MTF with perfect pixel
Frame Rate 60 fps
encapsulation

HDR
67 dB Single Exp • ASIL-B with functional safety
100 dB Double Exp

940 QE 7.5% Benefits

940 MTF 0.5 @ Nyq, >0.8 @ Nyq/2 • Enhance DMS Algorithms
CFA Options Mono, RGB-IR • HDR for optimal OMS
Package [mm] 10 x 8.5 iBGA

12
 VB1940

5.1 MP hybrid RGB-IR global and rolling shutter image sensor

Features
Parameter Specification Applications
• Global & rolling shutter mode
Pixel 2.25 um
• On-chip Bayerization • Hybrid DMS and OMS
Technology 3D stack BSI • HDR color images • Child presence and 2nd row
occupancy detection
Array Size 2560 x 1984, 1/2.5” • NIR smart upscale
Frame Rate 60 fps full resolution
• ASIL B Cybersecurity

GS: 61 dB
HDR
RS: 100 dB

940 QE 25%
Benefits
940 MTF > 0.5 @ Nyq/2
• Optimized GS for DMS algorithms
CFA Options RGB-IR • Enables cost effective ISPs
Package [mm] 10.3 x 8.9 OBGA • HDR for best color videos

13
 Leopard Imaging’s business

Professional
Design Services High Growth / High Value
Driver Development/ Vision Solutions
Image Tuning Services

• AI-enabled IOT Cameras

• Auto-grade Cameras
• Aerospace/Drone Cameras
• Robotic/AgTech Cameras
Provide ODM / OEM Services Customized Products • Edge Computing Cameras
for Global Customers Intelligent Vision Software • 2D/3D Depth Cameras
• Metaverse Cameras
• Hi-performing Machine Vision
• Hi-performance AI Box

14
 Leopard Imaging Differentiation
Wide Selections of Vision Products

Optical
Design Embedded Class 1000 Clean Room Multiple SMT Lines
Vision Multiple AA Machines
Specialist
Expertise

15 years of Deep Vision Knowledge

Image Quality Tuning Extensive Platform Vision-based AI Algorithm World class Operation
Knowhow & Software
15
 Leopard Imaging solutions for automotive

Robo Taxi/
Automotive Automotive
Last-mile EV Trucks/Buses Form Factor Processing Unit
Delivery

Support wiring distance beyond 15 meters Automotive grade solutions capability

Support front, side, rear, CMS and surround view Support 1MP - 8MP Cameras
Support DMS and OMS applications Support AA, lowlight, HDR, and LFM
IP67 and IP69K waterproofing Support SerDes, A-PHY, ASA, GigaE, & C2B

16
Leopard Imaging leveraging on ST image sensors

Need of high
MTF

Taking
advantage of
ST sensors to
build new class
of in-cabin
vision systems
Need highly
Need of high
innovative
sensitivity
features

17
 In-cabin camera system configuration
SerDes

Streamlined
L
e ST
n
s
Imager ECU
Control
LED
Driver

SerDes

ISP Integrated L
e ST
n
Advanced ECU
s
Imager
controller /
ISP
LED
Driver

SerDes
AI Integrated ISP
L
e ST
n
s
Imager AI SoC ECU
LED 18
Driver
 System design considerations
1 Reuse of current ECU convention Bayer-based ISP to support RGB-IR

RGB

Standalone ISP not needed

RGB-IR IR
ST sensor

2 Monocular 3D AI sensing RGB-IR sensor + AI SoC + AI in-cabin software

AI
SOC

ST RGB-IR Sensor Nextchip Eyeris Monocular

Indie 3D Sensing AI SW
19
 Optics design considerations
Visible 940nm

MTF
Quantum Efficiency

RGB-IR sensor
350 400 450 500 550 600 650 700 750 800 850 900 950 1000
Wavelength (nm) requires confocal
lens design RI
Quantum Efficiency

940 nm
Temp.
350 400 450 500 550 600 650 700 750 800 850 900 950 1000
Wavelength (nm) Drift
20
In-cabin camera solutions leveraging ST image sensors

LI-VB1940-VCL-OMS-FP3-137H

https://www.leopardimaging.com/dmscameras 21
Contacts
• Leopard Imaging – sales@leopardimaging.com Takeaways
• STMicroelectronics – Local Sales Representative

DMS and OMS systems are now becoming mandatory

1 Ensuring better safety for drivers and passenger
Enabling intuitive interactions with vehicles

ST has developed differentiated, competitive

2 automotive in-cabin image sensors using cutting edge
technologies with embedded cybersecurity

Leopard Imaging has extensive expertise in modules

3 Which leverage ST image sensors to help jumpstart your
design with solutions from 1M to 5M GS/RS cameras

22
 Find out more at Leopard Imaging Inc.
and Leopard Imaging - STMicroelectronics

© STMicroelectronics - All rights reserved.

ST logo is a trademark or a registered trademark of STMicroelectronics International NV or its affiliates in the EU and/or other countries.
For additional information about ST trademarks, please refer to www.st.com/trademarks.
All other product or service names are the property of their respective owners.