You are on page 1of 1

Foundation

Raspberry Pi
Documentation

Computers

Accessories

Microcontrollers

Services

Pico C SDK

Camera software

Introducing the Raspberry


Pi Cameras
Edit this on GitHub

There are now several o-cial Raspberry Pi camera


modules. The original 5-megapixel model was released in
2013, it was followed by an 8-megapixel Camera Module
2 which was released in 2016. The latest camera model is
the 12-megapixel Camera Module 3 which was released
in 2023. The original 5MP device is no longer available
from Raspberry Pi.

Additionally a 12-megapixel High Quality Camera with CS-


or M12-mount variants for use with external lenses was
released in 2020 and 2023 respectively. There is no
infrared version of the HQ Camera.

All of these cameras come in visible light and infrared


versions, while the Camera Module 3 also comes as a
standard or wide FoV model for a total of four different
variants.

Further details on the camera modules can be found in


the camera hardware page.

All Raspberry Pi cameras are capable of taking high-


resolution photographs, along with full HD 1080p video,
and can be fully controlled programmatically. This
documentation describes how to use the camera in
various scenarios, and how to use the various software
tools.

Once you’ve installed your camera module, there are


various ways the cameras can be used. The simplest
option is to use one of the provided camera applications,
such as libcamera-still or libcamera-vid.

libcamera and
libcamera-apps
Edit this on GitHub

Introduction
libcamera is a new software library aimed at supporting
complex camera systems directly from the Linux
operating system. In the case of the Raspberry Pi it
enables us to drive the camera system directly from open
source code running on ARM processors. The proprietary
code running on the Broadcom GPU, and to which users
have no access at all, is almost completely by-passed.

libcamera presents a C++ API to applications and works


at the level of conXguring the camera and then allowing
an application to request image frames. These image
buffers reside in system memory and can be passed
directly to still image encoders (such as JPEG) or to video
encoders (such as h.264), though such ancillary functions
as encoding images or displaying them are strictly
beyond the purview of libcamera itself.

For this reason Raspberry Pi supplies a small set of


example libcamera-apps. These are simple
applications, built on top of libcamera, and are designed
largely to emulate the function of the legacy stack built on
Broadcom’s proprietary GPU code (some users will
recognise these legacy applications as raspstill and
raspivid). The applications we provide are:

libcamera-hello A simple "hello world" application


which starts a camera preview stream and displays
it on the screen.

libcamera-jpeg A simple application to run a


preview window and then capture high resolution
still images.

libcamera-still A more complex still image capture


application which emulates more of the features of
raspistill.

libcamera-vid A video capture application.

libcamera-raw A basic application for capturing raw


(unprocessed Bayer) frames directly from the
sensor.

libcamera-detect This application is not built by


default, but users can build it if they have
TensorFlow Lite installed on their Raspberry Pi. It
captures JPEG images when certain objects are
detected.

Raspberry Pi’s libcamera-apps are not only command


line applications that make it easy to capture images and
video from the camera, they are also examples of how
users can create their own libcamera-based applications
with custom functionality to suit their own requirements.
The source code for the libcamera-apps is freely
available under a BSD 2-Clause licence at
https://github.com/raspberrypi/libcamera-apps.

More about libcamera

libcamera is an open source Linux community project.


More information is available at the libcamera website.

The libcamera source code can be found and checked


out from the o-cial libcamera repository, although we
work from a fork that lets us control when we get
libcamera updates.

Underneath the libcamera core, Raspberry Pi provides a


custom pipeline handler, which is the layer that
libcamera uses to drive the sensor and ISP (Image
Signal Processor) on the Raspberry Pi itself. Also part of
this is a collection of well-known control algorithms, or
IPAs (Image Processing Algorithms) in libcamera
parlance, such as AEC/AGC (Auto Exposure/Gain
Control), AWB (Auto White Balance), ALSC (Auto Lens
Shading Correction) and so on.

All this code is open source and now runs on the


Raspberry Pi’s ARM cores. There is only a very thin layer
of code on the GPU which translates Raspberry Pi’s own
control parameters into register writes for the Broadcom
ISP.

Raspberry Pi’s implementation of libcamera supports


not only the four standard Raspberry Pi cameras (the
OV5647 or V1 camera, the IMX219 or V2 camera, the
IMX477 or HQ camera and the IMX708 or Camera
Module 3) but also third party senors such as the IMX290,
IMX327, OV9281, IMX378. Raspberry Pi is keen to work
with vendors who would like to see their sensors
supported directly by libcamera.

Moreover, Raspberry Pi supplies a tuning =le for each of


these sensors which can be edited to change the
processing performed by the Raspberry Pi hardware on
the raw images received from the image sensor, including
aspects like the colour processing, the amount of noise
suppression or the behaviour of the control algorithms.

For further information on libcamera for the Raspberry


Pi, please consult the Tuning Guide for the Raspberry Pi
cameras and libcamera.

Getting Started

Using the camera for the Frst time

NOTE

On Raspberry Pi 3 and earlier devices running Bullseye


you need to re-enable Glamor in order to make the X-
Windows hardware accelerated preview window work.
To do this enter sudo raspi-config at a terminal
window and then choose Advanced Options,
Glamor and Yes. Finally quit raspi-config and let it
reboot your Raspberry Pi.

When running a Raspberry Pi OS based on Bullseye, the 5


basic libcamera-apps are already installed. In this case,
o-cial Raspberry Pi cameras will also be detected and
enabled automatically.

You can check that everything is working by entering:

libcamera-hello

You should see a camera preview window for about 5


seconds.

Users who are still running Buster should upgrade to


Bullseye. The new libcamera-based stack is no longer
supported there, and anyone still using Buster should stay
with the legacy camera stack.

NOTE

Raspberry Pi 3 and older devices may not by default


be using the correct display driver. Refer to the
/boot/config.txt Xle and ensure that either
dtoverlay=vc4-fkms-v3d or dtoverlay=vc4-
kms-v3d is currently active. Please reboot if you
needed to change this.

If you do need to alter the conFguration

You may need to alter the camera conXguration in your


/boot/config.txt Xle if:

You are using a 3rd party camera (the


manufacturer’s instructions should explain the
changes you need to make).

You are using an o-cial Raspberry Pi camera but


wish to use a non-standard driver/overlay.

If you do need to add your own dtoverlay, the following


are currently recognised.

Camera Module In /boot/config.txt

V1 camera (OV5647) dtoverlay=ov5647

V2 camera (IMX219) dtoverlay=imx219

HQ camera (IMX477) dtoverlay=imx477

GS camera (IMX296) dtoverlay=imx296

Camera Module 3 dtoverlay=imx708


(IMX708)

IMX290 and IMX327 dtoverlay=imx290,clock-


frequency=74250000 or
dtoverlay=imx290,clock-
frequency=37125000 (both
modules share the imx290
kernel driver; please refer to
instructions from the module
vendor for the correct
frequency)

IMX378 dtoverlay=imx378

OV9281 dtoverlay=ov9281

To override the automatic camera detection, Bullseye


users will also need to delete the entry
camera_auto_detect=1 if present in the config.txt
Xle. Your Raspberry Pi will need to be rebooted after
editing this Xle.

NOTE

Setting camera_auto_detect=0 disables the boot


time detection completely.

libcamera-hello
libcamera-hello is the equivalent of a "hello world"
application for the camera. It starts the camera, displays
a preview window, and does nothing else. For example

libcamera-hello

should display a preview window for about 5 seconds.


The -t <duration> option lets the user select how long
the window is displayed, where <duration> is given in
milliseconds. To run the preview indeXnitely, use:

libcamera-hello -t 0

The preview can be halted either by clicking the window’s


close button, or using Ctrl-C in the terminal.

Options

libcamera-apps uses a 3rd party library to interpret


command line options. This includes long form options
where the option name consists of more than one
character preceded by --, and short form options which
can only be a single character preceded by a single -. For
the most part option names are chosen to match those
used by the legacy raspicam applications with the
exception that we can no longer handle multi-character
option names with a single -. Any such legacy options
have been dropped and the long form with -- must be
used instead.

The options are classiXed broadly into 3 groups, namely


those that are common, those that are speciXc to still
images, and those that are for video encoding. They are
supported in an identical manner across all the
applications where they apply.

Please refer to the command line options documentation


for a complete list.

The Tuning File

Raspberry Pi’s libcamera implementation includes a


tuning =le for each different type of camera module. This
is a Xle that describes or "tunes" the parameters that will
be passed to the algorithms and hardware to produce the
best image quality. libcamera is only able to determine
automatically the image sensor being used, not the
module as a whole - even though the whole module
affects the "tuning".

For this reason it is sometimes necessary to override the


default tuning Xle for a particular sensor.

For example, the NOIR (no IR-Xlter) versions of sensors


require different AWB settings to the standard versions,
so the IMX219 NOIR should be run using

libcamera-hello --tuning-file /usr/share/libcamera/ipa/raspberrypi/imx219_noir.json

If you are using a Soho Enterprises SE327M12 module


you should use

libcamera-hello --tuning-file /usr/share/libcamera/ipa/raspberrypi/se327m12.json

Notice how this also means that users can copy an


existing tuning Xle and alter it according to their own
preferences, so long as the --tuning-file parameter is
pointed to the new version.

Finally, the --tuning-file parameter, in common with


other libcamera-hello command line options, applies
identically across all the libcamera-apps.

Preview Window

Most of the libcamera-apps display a preview image in


a window. When X Windows is not running it will draw
directly to the display using Linux DRM (Direct Rendering
Manager), otherwise it will attempt to use X Windows.
Both paths use zero-copy buffer sharing with the GPU,
and a consequence of this is that X forwarding is not
supported.

For this reason there is a third kind of preview window


which does support X forwarding, and can be requested
with the --qt-preview option. This implementation
does not beneXt from zero-copy buffer sharing nor from
3D acceleration which makes it computationally
expensive (especially for large previews), and so is not
normally recommended.

NOTE

Older systems using Gtk2 may, when linked with


OpenCV, produce Glib-GObject errors and fail to
show the Qt preview window. In this case please (as
root) edit the Xle /etc/xdg/qt5ct/qt5ct.conf and
replace the line containing style=gtk2 with
style=gtk3.

The preview window can be suppressed entirely with the


-n (--nopreview) option.

The --info-text option allows the user to request that


certain helpful image information is displayed on the
window title bar using "% directives". For example

libcamera-hello --info-text "red gain %rg, blue gain %bg"

will display the current red and blue gain values.

For the HQ camera, use --info-text "%focus" to


display the focus measure, which will be helpful for
focusing the lens.

A full description of the --info-text parameter is given


in the command line options documentation.

libcamera-jpeg
libcamera-jpeg is a simple still image capture
application. It deliberately avoids some of the additional
features of libcamera-still which attempts to
emulate raspistill more fully. As such the code is
signiXcantly easier to understand, and in practice still
provides many of the same features.

To capture a full resolution JPEG image use

libcamera-jpeg -o test.jpg

which will display a preview for about 5 seconds, and then


capture a full resolution JPEG image to the Xle test.jpg.

The -t <duration> option can be used to alter the


length of time the preview shows, and the --width and -
-height options will change the resolution of the
captured still image. For example

libcamera-jpeg -o test.jpg -t 2000 --width 640 --height 480

will capture a VGA sized image.

Exposure Control

All the libcamera-apps allow the user to run the camera


with Xxed shutter speed and gain. For example

libcamera-jpeg -o test.jpg -t 2000 --shutter 20000 --gain 1.5

would capture an image with an exposure of 20ms and a


gain of 1.5x. Note that the gain will be applied as
analogue gain within the sensor up until it reaches the
maximum analogue gain permitted by the kernel sensor
driver, after which the remainder will be applied as digital
gain.

Raspberry Pi’s AEC/AGC algorithm allows applications to


specify exposure compensation, that is, the ability to
make images darker or brighter by a given number of
stops, as follows

libcamera-jpeg --ev -0.5 -o darker.jpg


libcamera-jpeg --ev 0 -o normal.jpg
libcamera-jpeg --ev 0.5 -o brighter.jpg

Further remarks on Digital Gain

Digital gain is applied by the ISP (the Image Signal


Processor), not by the sensor. The digital gain will always
be very close to 1.0 unless:

The total gain requested (either by the --gain


option, or by the exposure proXle in the camera
tuning) exceeds that which can be applied as
analogue gain within the sensor. Only the extra gain
required will be applied as digital gain.

One of the colour gains is less than 1 (note that


colour gains are applied as digital gain too). In this
case the advertised digital gain will settle to 1 /
min(red_gain, blue_gain). This actually means that
one of the colour channels - just not the green one -
is having unity digital gain applied to it.

The AEC/AGC is changing. When the AEC/AGC is


moving the digital gain will typically vary to some
extent to try and smooth out any muctuations, but it
will quickly settle back to its "normal" value.

libcamera-still
libcamera-still is very similar to libcamera-jpeg
but supports more of the legacy raspistill options. As
before, a single image can be captured with

libcamera-still -o test.jpg

Encoders

libcamera-still allows Xles to be saved in a number


of different formats. It supports both png and bmp
encoding. It also allows Xles to be saved as a binary
dump of RGB or YUV pixels with no encoding or Xle
format at all. In these latter cases the application reading
the Xles will have to understand the pixel arrangement for
itself.

libcamera-still -e png -o test.png


libcamera-still -e bmp -o test.bmp
libcamera-still -e rgb -o test.data
libcamera-still -e yuv420 -o test.data

Note that the format in which the image is saved depends


on the -e (equivalently --encoding) option and is not
selected automatically based on the output Xle name.

Raw Image Capture

Raw images are the images produced directly by the


image sensor, before any processing is applied to them
either by the ISP (Image Signal Processor) or any of the
CPU cores. For colour image sensors these are usually
Bayer format images. Note that raw images are quite
different from the processed but unencoded RGB or YUV
images that we saw earlier.

To capture a raw image use

libcamera-still -r -o test.jpg

Here, the -r option (also --raw) indicates to capture the


raw image as well as the JPEG. In fact, the raw image is
the exact image from which the JPEG was produced.
Raw images are saved in DNG (Adobe Digital Negative)
format and are compatible with many standard
applications, such as dcraw or RawTherapee. The raw
image is saved to a Xle with the same name but the
extension .dng, thus test.dng in this case.

These DNG Xles contain metadata pertaining to the


image capture, including black levels, white balance
information and the colour matrix used by the ISP to
produce the JPEG. This makes these DNG Xles much
more convenient for later "by hand" raw conversion with
some of the aforementioned tools. Using exiftool
shows all the metadata encoded into the DNG Xle:

File Name : test.dng


Directory : .
File Size : 24 MB
File Modification Date/Time : 2021:08:17 16:36:18+01:00
File Access Date/Time : 2021:08:17 16:36:18+01:00
File Inode Change Date/Time : 2021:08:17 16:36:18+01:00
File Permissions : rw-r--r--
File Type : DNG
File Type Extension : dng
MIME Type : image/x-adobe-dng
Exif Byte Order : Little-endian (Intel, II)
Make : Raspberry Pi
Camera Model Name : /base/soc/i2c0mux/i2c@1/imx477@1a
Orientation : Horizontal (normal)
Software : libcamera-still
Subfile Type : Full-resolution Image
Image Width : 4056
Image Height : 3040
Bits Per Sample : 16
Compression : Uncompressed
Photometric Interpretation : Color Filter Array
Samples Per Pixel : 1
Planar Configuration : Chunky
CFA Repeat Pattern Dim : 2 2
CFA Pattern 2 : 2 1 1 0
Black Level Repeat Dim : 2 2
Black Level : 256 256 256 256
White Level : 4095
DNG Version : 1.1.0.0
DNG Backward Version : 1.0.0.0
Unique Camera Model : /base/soc/i2c0mux/i2c@1/imx477@1a
Color Matrix 1 : 0.8545269369 -0.2382823821 -0.09044229197 -0.1890484985 1.063961506 0.
As Shot Neutral : 0.4754476844 1 0.413686484
Calibration Illuminant 1 : D65
Strip Offsets : 0
Strip Byte Counts : 0
Exposure Time : 1/20
ISO : 400
CFA Pattern : [Blue,Green][Green,Red]
Image Size : 4056x3040
Megapixels : 12.3
Shutter Speed : 1/20

We note that there is only a single calibrated illuminant


(the one determined by the AWB algorithm even though it
gets labelled always as "D65"), and that dividing the ISO
number by 100 gives the analogue gain that was being
used.

Very long exposures

To capture very long exposure images, we need to be


careful to disable the AEC/AGC and AWB because these
algorithms will otherwise force the user to wait for a
number of frames while they converge. The way to
disable them is to supply explicit values. Additionally, the
entire preview phase of the capture can be skipped with
the --immediate option.

So to perform a 100 second exposure capture, use

libcamera-still -o long_exposure.jpg --
shutter 100000000 --gain 1 --awbgains 1,1 --
immediate

For reference, the maximum exposure times of the three


o-cial Raspberry Pi cameras can be found in this table.

libcamera-vid
libcamera-vid is the video capture application. By
default it uses the Raspberry Pi’s hardware H.264
encoder. It will display a preview window and write the
encoded bitstream to the speciXed output. For example,
to write a 10 second video to Xle use

libcamera-vid -t 10000 -o test.h264

The resulting Xle can be played with vlc (among other


applications)

vlc test.h264

Note that this is an unpackaged video bitstream, it is not


wrapped in any kind of container format (such as an mp4
Xle). The --save-pts option can be used to output frame
timestamps so that the bitstream can subsequently be
converted into an appropriate format using a tool like
mkvmerge.

libcamera-vid -o test.h264 --save-pts


timestamps.txt

and then if you want an mkv Xle:

mkvmerge -o test.mkv --timecodes


0:timestamps.txt test.h264

Encoders

There is support for motion JPEG, and also for


uncompressed and unformatted YUV420, for example

libcamera-vid -t 10000 --codec mjpeg -o test.mjpeg


libcamera-vid -t 10000 --codec yuv420 -o test.data

In both cases the --codec parameter determines the


output format, not the extension of the output Xle.

The --segment parameter breaks output Xles up into


chunks of the segment size (given in milliseconds). This
is quite handy for breaking a motion JPEG stream up into
individual JPEG Xles by specifying very short (1
millisecond) segments.

libcamera-vid -t 10000 --codec mjpeg --segment 1 -o test%05d.jpeg

Observe that the output Xle name is normally only


sensible if we avoid over-writing the previous Xle every
time, such as by using a Xle name that includes a counter
(as above). More information on output Xle names is
available below.

Network Streaming

NOTE

You might also like