Insight from Interactions 1

Screen Capture
Introduction to
screen capture.



September 20, 2005
By Zigler Nir
Insight from Interactions 2
Agenda
 What is Screen Capture?
 ScreenAgent – required characteristics
 Capture Methods
 Screen Capture Stream
 Screen Components
 ScreenAgent CAPI Registration
 ScreenAgent Installation
 ScreenAgent in a Citrix/Terminal-Server Environment
 ScreenAgent Limitations



Insight from Interactions 3
Screen Capture
 Screen Capture - What is it?
Capturing screen activity that is taking place on a screen,
transporting it to a server for storage, and enabling playback of the
captured data upon request.



 Screen Capture - Why do we need it?
In our digital world, listening to a voice only recording is not enough
to fully experience the interaction that took place.
Insight from Interactions 4
Screen Capture – Requested Characteristics
 Transparent & undetectable screen capture.
 Minimal footprint.
 Compatibility.
 Ease of installation (mass-deployment support).
 Capture quality.

Insight from Interactions 5
ScreenAgent Effectiveness Metrics
There are four inter-connected measurements for ScreenAgent
effectiveness:

 Component compliance.
 Data rate.
 Playback quality.
 Host performance.
Insight from Interactions 6
How to capture screen?
 This is the most simple and naïve method
 How we can improve it?

Insight from Interactions 7
Screen Capture – Scraper Method


Insight from Interactions 8
Screen Capture – Scraper Method


Scraper Pointer
Scraper Pointer
Scraper Pointer
Insight from Interactions 9
Scraper Method – Summary
 The screen is virtually divided to rectangles.
 The rectangles are periodically scanned for changes (left to right, top
to bottom).
 Rectangles that a change occurred in their boundary will be
recaptured.
 Optimization for capturing the cursor.
Insight from Interactions 10
Screen Capture – Hooking Method


Applications
GDI Hook
Screen
Virtual
Screen
Insight from Interactions 11
Hooking Method – Summary
 A hook is made to the OS GDI layer.
 Each GDI command is duplicated and sent to the capture component
as well as to the video card.
 The method of “hooking” to the GDI is different between OS
platforms:
– Win9x/WinME - Win32 Hook to the GDI32.dll
– WinNT - Replacing the video kernel driver.
– Win2k/WinXP - Usage of a Mirror Driver.

Insight from Interactions 12
Hooking Method – Summary
Windows Applications
Tb2Wdg95
GDI
Video Driver
U
s
e
r

M
o
d
e

Windows Applications
Tb2Wdg40
GDI
Video Driver
U
s
e
r

M
o
d
e

K
e
r
n
e
l

M
o
d
e

K
e
r
n
e
l

M
o
d
e

U
s
e
r

M
o
d
e

Windows Applications
Tb2Mirror
GDI
Video Driver
Win9X/WinME WinNT Win2K/WinXP
Insight from Interactions 13
Screen Capture Stream
The ScreenAgent capture stream is composed from Packets. Each
packet can be from one of the following types:

 Key Frame (KF) [ ]
Key Frame packets represent a capture of the entire screen.

 Delta [ ]
Delta packets represent the capture difference from the previous
packet.
Insight from Interactions 14
Screen Capture Stream - Example
KF


Delta’s


t
. . . . . . .
KF


t
KF


KF


Insight from Interactions 15
Components Participating In the Record


CLS Server
Screen
Logger
Workstations with
ScreenAgent
Registration – I’m alive
Start Record
for Agent X
Start Record
Captured Data
Insight from Interactions 16
The Record Process – Inside View


Workstation
Screen
Logger
Capture
Engine
Comm
Module
ScreenAgent Process
Packet
Throttle Buffer
Packet Packet
Insight from Interactions 17
ScreenAgent CAPI Registration
Parameters sent to the CAPI upon registration:
 GUID
 IP Address (IP Address:Port)
 Machine Name
 OS Login
 Extension+SwitchId pairs
or
AgentId+SwitchId pairs
 OPTIONAL
Insight from Interactions 18
Registration in a Free Sitting Env
Database
User Admin

Name = Nir Zigler
AgentId = 2982
.
.
ScreenAgent
Ext=3625, Sw=1
CLS
Ext=3625
Agent Nir logs-in to Ext 3625
using AgentId 2982
AgentId 2982 in Ext
3625

Login Table
Register
IP=192.168.10.1
Ext=3625, Sw=1
Universal
Agent Table
PBX
Incoming Call
for Ext=3625
Register
StartRecord
Insight from Interactions 19
Registration in a Free PC Env
Database
User Admin

Name = Nir Zigler
AgentId = 2982
OSLogin =Nirz
.
ScreenAgent
CLS
Ext=3625
Agent Nir logs-in to Ext 3625
using AgentId 2982
AgentId 2982 in Ext
3625

Login Table
Register
IP=192.168.10.1
OSLogin=Nirz
Universal
Agent Table
PBX
Incoming Call
for Ext=3625
Register
StartRecord
Insight from Interactions 20
ScreenAgent Installation
 Interactive Installation
– The installer navigates through a set of windows and supply information required
for the installation.
– Mandatory Information that is required during the installation include: CLS IP
address, Screen Reporting IP address, Identification mode.

 Silent Installation
– ScreenAgent is installed without user interaction.
– ScreenAgent setup enables silent installation, it doesn’t perform push deployment.
– Mandatory information is supplied to the installation though the usage of the
Agent.cfg file.

 Upgrades
The purpose of an upgrade is to reinstall a new set of binaries while retaining the
current configuration.


Insight from Interactions 21
Installation Switches
 Setup.exe -NICE_SHUT
Reboots the workstation at the end of the installation (on WinNT, Win2K, WinXP).

 Setup.exe -UNIQUE_AGENT_USERNAME
Invoke silent installation with Unique OS Login registration mode.

 Setup.exe -WS_INFO:E1234,S1; E9876,S2
Invoke silent installation with Extension registration mode.
E = Extension’s
A = AgentId’s
S = SwitchId’s

 Setup.exe –OLD_WS_INFO
Performs silent upgrade.


Insight from Interactions 22
Citrix/Terminal Server
 What is Terminal Server?
An environment were all the applications are installed and run from a centralize server. The
user connects to the server with a “dumb terminal” that send keyboard strokes & mouse
movements as Input to the server and receives screen display as output.

How does it works?
– When a user connects to the server, a new session is created.
– A session represents a so called “virtual computer” that is dedicated to the user.

Citrix Advantages
– Optimized protocol between the Server and the Clients.
– Clients can run on any OS platforms.
– Supports Application Mode that enables load balancing.

Insight from Interactions 23
Citrix/Terminal Server
Word
Calc
Session #1
UserName: NIR
Excel
Word
Session #2
UserName: EFRAT
User NIR connects to
the Server
User EFRAT connects
to the Server
SA
SA
Insight from Interactions 24
Screen in a Citrix/Terminal Server Env
 The ScreenAgent is installed on the server (Once per server).
 Scraper is the only available capture method in a Citrix/Terminal
Server environment.
 To reduce CPU/Memory usage – capture component and record
buffer are initialized only on StartRecord request.
 In a Citrix/Terminal Server Env the ScreenAgent port consists of
BasePort + SessionId
Example: Server IP is 192.178.20.20, BasePort is 3020.
First ScreenAgent IP is : 192.178.20.20:3021
Second ScreenAgent IP is : 192.178.20.20:3022
Insight from Interactions 25
ScreenAgent Limitations
 ScreenAgent can’t capture “accelerated” graphics (i.e. DirectX,
OpenGL).
 ScreenAgent can’t capture Full Dos windows.
 ScreenAgent process is protected from termination on all OS platform
except on WinXP when the user has Administrator privileges.
 When installed on WinNT with another remote-control application the
ScreenAgent must be installed in scraper only mode.
Insight from Interactions 26
Q
A
&
Insight from Interactions 27
THANK YOU
FOR JOINING
US TODAY
www.nice.com