How does OpenGL work at the lowest level?

[closed]
Asked 12 years, 9 months ago Modified 8 years, 1 month ago Viewed 43k times

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or
rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it
97 can be reopened, visit the help center.

Closed 12 years ago.

The community reviewed whether to reopen this question 1 year ago and left it closed:

Original close reason(s) were not resolved

I understand how to write OpenGL/DirectX programs, and I know the maths and the conceptual stuff
behind it, but I'm curious how the GPU-CPU communication works on a low level.

Say I've got an OpenGL program written in C that displays a triangle and rotates the camera by 45
degrees. When I compile this program, will it be turned into a series of ioctl-calls, and the gpu driver then
sends the appropriate commands to the gpu, where all the logic of rotating the triangle and setting the
appropriate pixels in the appropriate color is wired in? Or will the program be compiled into a "gpu
program" which is loaded onto the gpu and computes the rotation etc.? Or something completely
different?

Edit: A few days later I found this article series, which basically answers the question:
http://fgiesen.wordpress.com/2011/07/01/a-trip-through-the-graphics-pipeline-2011-part-1/

opengl gpu

Share Improve this question Follow edited Jul 4, 2011 at 0:01 asked Jun 19, 2011 at 0:05
Benno
5,376 5 42 63

OpenGL specification state that commands are sent in a client/server mode. The specification is too broad to
determine a specific tecnology used by OpenGL implementations. – Luca Jun 19, 2011 at 18:21

4 Answers Sorted by: Highest score (default)

This question is almost impossible to answer because OpenGL by itself is just a front end API, and as
long as an implementations adheres to the specification and the outcome conforms to this it can be
97 done any way you like.

The question may have been: How does an OpenGL driver work on the lowest level. Now this is again
impossible to answer in general, as a driver is closely tied to some piece of hardware, which may again
do things however the developer designed it.

So the question should have been: "How does it look on average behind the scenes of OpenGL and the
graphics system?". Let's look at this from the bottom up:
 1. At the lowest level there's some graphics device. Nowadays these are GPUs which provide a set of
registers controlling their operation (which registers exactly is device dependent) have some
program memory for shaders, bulk memory for input data (vertices, textures, etc.) and an I/O
channel to the rest of the system over which it recieves/sends data and command streams.

2. The graphics driver keeps track of the GPUs state and all the resources application programs that
make use of the GPU. Also it is responsible for conversion or any other processing the data sent by
applications (convert textures into the pixelformat supported by the GPU, compile shaders in the
machine code of the GPU). Furthermore it provides some abstract, driver dependent interface to
application programs.

3. Then there's the driver dependent OpenGL client library/driver. On Windows this gets loaded by
proxy through opengl32.dll, on Unix systems this resides in two places:
X11 GLX module and driver dependent GLX driver

and /usr/lib/libGL.so may contain some driver dependent stuff for direct rendering

On MacOS X this happens to be the "OpenGL Framework".

It is this part that translates OpenGL calls how you do it into calls to the driver specific functions in
the part of the driver described in (2).

4. Finally the actual OpenGL API library, opengl32.dll in Windows, and on Unix /usr/lib/libGL.so; this
mostly just passes down the commands to the OpenGL implementation proper.

How the actual communication happens can not be generalized:

In Unix the 3<->4 connection may happen either over Sockets (yes, it may, and does go over network if
you want to) or through Shared Memory. In Windows the interface library and the driver client are both
loaded into the process address space, so that's no so much communication but simple function calls
and variable/pointer passing. In MacOS X this is similar to Windows, only that there's no separation
between OpenGL interface and driver client (that's the reason why MacOS X is so slow to keep up with
new OpenGL versions, it always requires a full operating system upgrade to deliver the new framework).

Communication betwen 3<->2 may go through ioctl, read/write, or through mapping some memory into
process address space and configuring the MMU to trigger some driver code whenever changes to that
memory are done. This is quite similar on any operating system since you always have to cross the
kernel/userland boundary: Ultimately you go through some syscall.

Communication between system and GPU happen through the periphial bus and the access methods it
defines, so PCI, AGP, PCI-E, etc, which work through Port-I/O, Memory Mapped I/O, DMA, IRQs.

Share Improve this answer Follow edited Jun 19, 2011 at 12:06 answered Jun 19, 2011 at 9:24
datenwolf
161k 13 189 303

1 This description is probably too low-level and generic with very little OpenGL specifics. It roughly applies to any
hardware component present in a computer. – v.shashenko Jul 15, 2016 at 15:40

2 @v.shashenko: Well, of course. Regardless of which hardware component you're talking to, it follows this general
scheme. But OpenGL is not a particular piece of software, it's just a specification and anything that conforms to the
specification is a valid implementation. And because in the end a OpenGL implementation is talking to a GPU it will
roughly follow the same script along which hardware facing APIs are implemented. It's impossible to become any
more specific than that, because there is not "The ONE" OpenGL implementation. And every implementation is a
little bit different. – datenwolf Jul 15, 2016 at 15:55

1 @v.shashenko: If you want more details, then you'd have to ask about a particular implementation. For example
say, X11+GLX+DRI → Mesa/AMDGPU → Linux-KMS+DRM(Direct Rendering Manager). And each of these
 components is a rather complex beast with so many details, that you could fill books with details on how each
component works. But that would describe the Linux-AMDGPU implementation. But Linux+NVidia is an entirely
different beast. And on Windows its different again. – datenwolf Jul 15, 2016 at 15:59

Do you remember me we chat for long few days back , i just wanted to know how to contact you in case i want to
know something.? – Suraj Jain Sep 17, 2016 at 17:33

What you taught that day , cleared lot of my doubts , made some new one too , but overall now i do not consider it
a magic that we draw windows and icon on screen. – Suraj Jain Sep 17, 2016 at 17:34

When I compile this program, will it be turned into a series of ioctl-calls, and the gpu driver then
sends the appropriate commands to the gpu, where all the logic of rotating the triangle and
24 setting the appropriate pixels in the appropriate color is wired in? Or will the program be
compiled into a "gpu program" which is loaded onto the gpu and computes the rotation etc.?

You're not far off. Your program calls the installable client driver (which is not really a driver, it's a
userspace shared library). That will use ioctl or a similar mechanism to pass data to the kernel driver.

For the next part, it depends on the hardware. Older video cards had what is called a "fixed-function
pipeline". There were dedicated memory spaces in the video card for matrices, and dedicated hardware
for texture lookup, blending, etc. The video driver would load the right data and flags for each of these
units and then set up DMA to transfer your vertex data (position, color, texture coordinates, etc).

Newer hardware has processor cores ("shaders") inside the video card, which differ from your CPU in
that they each run much slower, but there are many more of them working in parallel. For these video
cards, the driver prepares program binaries to run on the GPU shaders.

Share Improve this answer Follow edited Mar 10, 2016 at 17:48 answered Jun 19, 2011 at 1:55
Ben Voigt
281k 44 426 728

Your program is not compiled for any particular GPU; it is just dynamically linked against a library that
will implement OpenGL. The actual implementation might involve sending OpenGL commands to the
22 GPU, running software fallbacks, compiling shaders and sending them to the GPU, or even using
shader fallbacks to OpenGL commands. The graphics landscape is fairly complicated. Thankfully linking
insulates you from most of the drivers' complexity, leaving driver implementers free to use whatever
techniques they see fit.

Share Improve this answer Follow answered Jun 19, 2011 at 0:48
Tobu
25k 4 92 98

C/C++ compilers/linkers do exactly one thing: they convert text files into a series of machine-specific
opcodes that are run on the CPU. OpenGL and Direct3D are just C/C++ APIs; they cannot magically
20 convert your C/C++ compiler/linker into a compiler/linker for the GPU.

Every line of C/C++ code you write will be executed on the CPU. Calls to OpenGL/Direct3D will call into
C/C++ libraries, static or dynamic as the case may be.

The only place when a "gpu program" would come into play is if your code explicitly creates shaders.
That is, if you make the API calls into OpenGL/D3D that cause the compiling and linking of shaders. To
 do this, you (at runtime, not C/C++ compile-time) either generate or load strings that represent shaders
in some shader language. You then shove them through the shader compiler, and get back an object in
that API that represents that shader. You then apply one or more shaders to a particular rendering
command. Each of these steps happen explicitly at the direction of your C/C++ code, which as
previously stated runs on the CPU.

Many shader languages use C/C++-like syntax. But that doesn't make them equivalent to C/C++.

Share Improve this answer Follow answered Jun 19, 2011 at 0:31
Nicol Bolas
462k 63 799 1k

I've been learning opengl as of late and have been wonder why I see all these "shader programs" in code I look at
written as litteral strings. Like you said, they are compiled at run time. I saw some examples in some android
opengl es code. Then I saw some code for iphone too. Apparently the iPhone has four vector processors that are
able to do floating point math much faster than the iPhone's CPU can -- so you can write the ASM as literal strings
to get compiled at run time to use those processors instead of the main cpu. – eggie5 Jun 19, 2011 at 1:02

1 Most of the "fixed function pipeline" is now implemented by a default set of shader programs. You don't need to
explicitly request a shader to get one. – Ben Voigt Jun 19, 2011 at 1:56

1 @Ben Voigt: True, but if nobody told you, you wouldn't be able to tell the difference. – Nicol Bolas Jun 19, 2011 at
2:54

1 Which might be a sensible attitude to take, if this question weren't about the hidden details. – Ben Voigt Jun 19,
2011 at 4:17

Highly active question. Earn 10 reputation (not counting the association bonus) in order to answer this question. The
reputation requirement helps protect this question from spam and non-answer activity.