Professional Documents
Culture Documents
Android Multimedia Video: Assignment: 1 Date:27/11/2020
Android Multimedia Video: Assignment: 1 Date:27/11/2020
Source: https://androidxref.com/
1) Application Framework
Basically application code utilize android.media APIs to interact with the successive layer through java
native interface.
Provides classes that manage various media interfaces in audio and video.
The Media APIs are used to play and, in some cases, record media files.
e.g., play a video streamed over the web or from local storage
2) Binder IPC
Location : frameworks/av/media/libmedia
MediaPlayer class can be used to control playback of audio/video files and streams.
e.g., on how to use the methods in this class can be found in VideoView
MediaCodec.BufferInfo Per buffer metadata includes an offset and size specifying the
range of valid data in the associated codec (output) buffer.
MediaCodec.Callback
MediaCodec callback interface. Used to notify the user asynchronously of various MediaCodec
events.
Location : /frameworks/av/media/libmedia/include/media/mediarecorder.h
Media Recorder class is to record audio/video streams form device and setting
various parameters.
Location: /frameworks/av/media/libmedia/mediarecorder.cpp
setCamera Checks media recorder state (initialized) and set camera for recording.
Eg: mMediaRecorder->setCamera(camera, proxy)
setPreviewSurface try to set preview surface with setting the video source.
setVideoSource set video source i.e (mMediaRecorder->setVideoSource(vs).
setAudioSource initialize media recorder and sets audio source.
setOutputFormat output format for audio recording is set.
setVideoEncoder video encoding format to the source is assigned here.
At the native level, Android provides a multimedia framework that utilizes the Stagefright engine for
audio and video recording and playback. Stage fright comes with a default list of supported software
codecs.
I) MediaPlayerService
class is defined in MediaPlayerService.h and definitions are implemented in MediaPlayerService.cpp.
PATH: /frameworks/av/media/libmediaplayerservice/MediaPlayerService
MediaPlayerService
This class is primary API for providing services to video and sound. It creates player, MediaRecorder ,
remove Media recorder Client, creates Meta data retriever, display codec list.
AudioOutput provide audio services to Client based on device callback, set audio stream type,
playback rate, audio track, audio attributes, sample rates and volume.
CallbackData is passed to AudioTrack as user data. tryBeginTrackSwitch/endTrackSwitch are used
when the CallbackData is handed over to next sink.
ImediaPlayerService is an interface. It creates/remove Media Recorder Client and prepare codec
list.
BatteryUsageInfo API for battery app to pull data of codecs usage and collect information of
codec usage from media player and media recorder.
BatteryAudioFlingerUsageInfo count the number of audio streams played, display total time of
audio output device usage and check whether device is currently in use or not.
Client : This API create player and set video surface texture, buffering settings , sync settings , set
volume, set audio stream type, set metadata, get current position and duration, reset and notify,
set next player. It also provide services to client like start, stop, pause, isplaying
AudioDeviceUpdatedNotifier sends notification to client from class Listener.
MediaRecorderClient
This API checks camera permissions to set video source and set audio and camera sources.
Provides the check service for camera if we don’t know it exists. Set the size and format rate of
video.
ServiceDeathNotifier class notifies the client about binderDied and serviceDied.
AudioDeviceUpdateNotifier sends the notification to client about audio io handle and audio port
handle from class Listener.
MetadataRetrieverClient
This API implements ImediaMetadataRetriever interface and extract Meta data. It also create Retriever
by setting data source parameters like offset and length from Media player factory.
File Formats: 3GPP (.3gp), MPEG-4 (.mp4), Matroska (.mkv) ,MPEG-TS (.ts), WebM (.webm)
For video Encoding, Decoding, Playback & Streaming speed and resolution are recommended based on
the certain parameters.
e.g.: (SD- Low/High) quality Format (H.264) -> (176 x 144 px @ 12 fps ) (480 x 360 px @ 30 fps) are
recommended respectively.
>> Similarly, for images also Android framework recommends some formats.
Stage fright is a native media playback tool used by Android and all these weaknesses reside in it.
Extractor: extract data into required format and set communication between native media
server to kernel driver.
Player: captures video and provide sources like playing, starting, etc.
Codex: prepare playback and setup decoder on request from media service to kernel driver.
PATH: /frameworks/av/media/libstagefright/
4) OMX INTEGRATION
The OpenMAX IL provides a standardized way for Stagefright to recognize and use custom hardware-
based multimedia codecs. Its acts as an interface between the Stagefright Engine and the codec.
Path-framerwork/av/media/libmedia/IOMX.cpp
Class-BpOMX
Path-framerwork/av/media/libstagefright/omx/omxgraphicbuffersource.cpp
Class-OmxComponentWrapper
Path-framerwork/av/media/libstagefright/omx/OMXmaster.cpp
OMXmaster.cpp Used for:-
1)addPlugin
2)clearPlugins
3)makeComponentInstance
4)destroyComponentInstance
5)enumerateComponents
6)getRolesOfComponent
OMXutils:
Path-framerwork/av/media/libstagefright/omx/OMXutils.cpp
Used for:-
1.DescribeColorFormatParams
2.DescribeDefaultColorFormat
3.GetComponentRole