P. 1
CUDA FORTRAN

CUDA FORTRAN

|Views: 4,034|Likes:
Published by Mark Munthe

More info:

Published by: Mark Munthe on May 18, 2011
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

06/12/2013

pdf

text

original

Sections

  • CUDA Fortran Kernels
  • Thread Blocks
  • Memory Hierarchy
  • Subroutine / Function Qualifiers
  • Attributes(host)
  • Attributes(global)
  • Attributes(device)
  • Restrictions
  • Variable Qualifiers
  • Attributes(constant)
  • Attributes(shared)
  • Attributes(pinned)
  • Datatypes in Device Subprograms
  • Predefined Variables in Device Subprograms
  • Execution Configuration
  • Asynchronous concurrent execution
  • Concurrent Host and Device Execution
  • Concurrent Stream Execution
  • Kernel Loop Directive
  • Restrictions on the CUF kernel directive
  • Building a CUDA Fortran Program
  • Emulation Mode
  • New Subroutine and Function Attributes
  • Host Subroutines and Functions
  • Global Subroutines
  • Device Subroutines and Functions
  • Restrictions on Device Subprograms
  • Variable Attributes
  • Device data
  • Constant data
  • Shared data
  • Value dummy arguments
  • Pinned arrays
  • Allocating Device and Pinned Arrays
  • Allocating Device Memory
  • Allocating Device Memory Using Runtime Routines
  • Allocating Pinned Memory
  • Data transfer between host and device memory
  • Data Transfer Using Assignment Statements
  • Implicit Data Transfer in Expressions
  • Data Transfer Using Runtime Routines
  • Invoking a kernel subroutine
  • Device code
  • Datatypes allowed
  • Builtin variables
  • Fortran intrinsics
  • Fortran Numeric and Logical Intrinsics
  • Fortran Mathematical Intrinsics
  • Fortran Numeric Inquiry Intrinsics
  • Fortran Bit Manipulation Intrinsics
  • Fortran Reduction Intrinsics
  • Fortran Random Number Intrinsics
  • New Intrinsic Functions
  • Warp-Vote Operations
  • Atomic Functions
  • Arithmetic and Bitwise Atomic Functions
  • Counting Atomic Functions
  • Host code
  • SIZEOF Intrinsic
  • Initialization
  • Device Management
  • cudaGetDeviceCount
  • cudaSetDevice
  • cudaSetDeviceFlags
  • cudaGetDevice
  • cudaGetDeviceProperties
  • cudaChooseDevice
  • Thread Management
  • cudaThreadSynchronize
  • cudaThreadExit
  • Memory Management
  • cudaMalloc
  • cudaMallocPitch
  • cudaFree
  • cudaMallocArray
  • cudaFreeArray
  • cudaMemset
  • cudaMemset2D
  • cudaMemcpy
  • cudaMemcpyAsync
  • cudaMemcpy2D
  • cudaMemcpy2DAsync
  • cudaMemcpyToArray
  • cudaMemcpy2DToArray
  • cudaMemcpyFromArray
  • cudaMemcpy2DFromArray
  • cudaMemcpyArrayToArray
  • cudaMemcpy2DArrayToArray
  • cudaMalloc3D
  • cudaMalloc3DArray
  • cudaMemset3D
  • cudaMemcpy3D
  • cudaMemcpy3DAsync
  • cudaMemcpyToSymbol
  • cudaMemcpyFromSymbol
  • cudaMemcpyToSymbolAsync
  • cudaMemcpyFromSymbolAsync
  • cudaGetSymbolAddress
  • cudaGetSymbolSize
  • cudaHostAlloc
  • cudaHostGetDevicePointer
  • cudaMallocHost
  • cudaFreeHost
  • Stream Management
  • cudaStreamCreate
  • cudaStreamQuery
  • cudaStreamSynchronize
  • cudaStreamDestroy
  • Event Management
  • cudaEventCreate
  • cudaEventRecord
  • cudaEventQuery
  • cudaEventSynchronize
  • cudaEventDestroy
  • cudaEventElapsedTime
  • Error Handling
  • cudaGetLastError
  • cudaGetErrorString
  • Version Management
  • cudaDriverGetVersion
  • cudaRuntimeGetVersion
  • Matrix Multiplication Example
  • Source Code Listing
  • Source Code Description

CUDA Fortran

Programming Guide and Reference

Release 2011

The Portland Group

While every precaution has been taken in the preparation of this document, The Portland Group® (PGI®), a wholly-owned subsidiary of STMicroelectronics, Inc., makes no warranty for the use of its products and assumes no responsibility for any errors that may appear, or for damages resulting from the use of the information contained herein. The Portland Group retains the right to make changes to this information at any time, without notice. The software described in this document is distributed under license from STMicroelectronics and/or The Portland Group and may be used or copied only in accordance with the terms of the license agreement ("EULA"). PGI Workstation, PGI Server, PGI Accelerator, PGF95, PGF90, PGFORTRAN, and PGI Unified Binary are trademarks; and PGI, PGHPF, PGF77, PGCC, PGC++, PGI Visual Fortran, PVF, PGI CDK, Cluster Development Kit, PGPROF, PGDBG, and The Portland Group are registered trademarks of The Portland Group Incorporated. No part of this document may be reproduced or transmitted in any form or by any means, for any purpose other than the purchaser's or the end user's personal use without the express written permission of STMicroelectronics and/or The Portland Group.

Copyright © 2010 The Portland Group® and STMicroelectronics, Inc.All rights reserved. Printed in the United States of America Release 2010, version 10.8, August 2010 Release 2010, version 10.9, September 2010 Release 2011, version 11.0, November 2010

Technical support: trs@pgroup.com Sales: sales@pgroup.com Web: www.pgroup.com

ID: 103151117

Contents
Preface ...................................................................................................................................... ix
Intended Audience ................................................................................................................. ix Organization .......................................................................................................................... ix Conventions ........................................................................................................................... ix Terminology ........................................................................................................................... x Related Publications ................................................................................................................ x

1. Introduction ........................................................................................................................ 1 2. Programming Guide .......................................................................................................... 3
CUDA Fortran Kernels .............................................................................................................. 3 Thread Blocks ........................................................................................................................ 4 Memory Hierarchy .................................................................................................................. 4 Subroutine / Function Qualifiers ................................................................................................ 4 Attributes(host) .............................................................................................................. 5 Attributes(global) ............................................................................................................ 5 Attributes(device) ........................................................................................................... 5 Restrictions .................................................................................................................... 5 Variable Qualifiers .................................................................................................................. 5 Attributes(device) ........................................................................................................... 5 Attributes(constant) ........................................................................................................ 6 Attributes(shared) ........................................................................................................... 6 Attributes(pinned) .......................................................................................................... 6 Datatypes in Device Subprograms ............................................................................................. 6 Predefined Variables in Device Subprograms .............................................................................. 7 Execution Configuration ........................................................................................................... 7 Asynchronous concurrent execution .......................................................................................... 8 Concurrent Host and Device Execution .............................................................................. 8 Concurrent Stream Execution ........................................................................................... 8 Kernel Loop Directive .............................................................................................................. 8 Restrictions on the CUF kernel directive ........................................................................... 10 Building a CUDA Fortran Program ........................................................................................... 11 Emulation Mode ................................................................................................................... 11 iii

3. Reference ........................................................................................................................... 13
New Subroutine and Function Attributes ................................................................................... Host Subroutines and Functions ...................................................................................... Global Subroutines ........................................................................................................ Device Subroutines and Functions ................................................................................... Restrictions on Device Subprograms ................................................................................ Variable Attributes ................................................................................................................. Device data .................................................................................................................. Constant data ................................................................................................................ Shared data .................................................................................................................. Value dummy arguments ................................................................................................ Pinned arrays ............................................................................................................... Allocating Device and Pinned Arrays ........................................................................................ Allocating Device Memory .............................................................................................. Allocating Device Memory Using Runtime Routines ............................................................ Allocating Pinned Memory .............................................................................................. Data transfer between host and device memory ......................................................................... Data Transfer Using Assignment Statements ....................................................................... Implicit Data Transfer in Expressions ............................................................................... Data Transfer Using Runtime Routines ............................................................................. Invoking a kernel subroutine .................................................................................................. Device code ......................................................................................................................... Datatypes allowed ......................................................................................................... Builtin variables ............................................................................................................ Fortran intrinsics .......................................................................................................... New Intrinsic Functions ................................................................................................. Warp-Vote Operations .................................................................................................... Atomic Functions .......................................................................................................... Restrictions .................................................................................................................. Host code ............................................................................................................................ SIZEOF Intrinsic ............................................................................................................ 13 13 13 14 14 14 14 15 16 17 17 17 17 18 18 19 19 19 20 20 21 21 22 22 24 26 27 29 29 29

4. Runtime APIs .................................................................................................................... 31
Initialization ......................................................................................................................... Device Management ............................................................................................................... cudaGetDeviceCount ...................................................................................................... cudaSetDevice ............................................................................................................... cudaSetDeviceFlags ........................................................................................................ cudaGetDevice .............................................................................................................. cudaGetDeviceProperties ................................................................................................ cudaChooseDevice ......................................................................................................... Thread Management .............................................................................................................. cudaThreadSynchronize ................................................................................................. cudaThreadExit ............................................................................................................. Memory Management ............................................................................................................ cudaMalloc .................................................................................................................. iv 31 31 31 31 31 32 32 32 32 32 32 32 33

cudaMallocPitch ............................................................................................................ cudaFree ...................................................................................................................... cudaMallocArray ........................................................................................................... cudaFreeArray .............................................................................................................. cudaMemset ................................................................................................................. cudaMemset2D ............................................................................................................. cudaMemcpy ................................................................................................................ cudaMemcpyAsync ........................................................................................................ cudaMemcpy2D ............................................................................................................ cudaMemcpy2DAsync .................................................................................................... cudaMemcpyToArray ...................................................................................................... cudaMemcpy2DToArray .................................................................................................. cudaMemcpyFromArray .................................................................................................. cudaMemcpy2DFromArray .............................................................................................. cudaMemcpyArrayToArray .............................................................................................. cudaMemcpy2DArrayToArray .......................................................................................... cudaMalloc3D .............................................................................................................. cudaMalloc3DArray ....................................................................................................... cudaMemset3D ............................................................................................................. cudaMemcpy3D ............................................................................................................ cudaMemcpy3DAsync .................................................................................................... cudaMemcpyToSymbol ................................................................................................... cudaMemcpyFromSymbol ............................................................................................... cudaMemcpyToSymbolAsync ........................................................................................... cudaMemcpyFromSymbolAsync ....................................................................................... cudaGetSymbolAddress ................................................................................................... cudaGetSymbolSize ........................................................................................................ cudaHostAlloc ............................................................................................................... cudaHostGetDevicePointer .............................................................................................. cudaMallocHost ............................................................................................................ cudaFreeHost ................................................................................................................ Stream Management .............................................................................................................. cudaStreamCreate .......................................................................................................... cudaStreamQuery .......................................................................................................... cudaStreamSynchronize .................................................................................................. cudaStreamDestroy ........................................................................................................ Event Management ................................................................................................................ cudaEventCreate ............................................................................................................ cudaEventRecord .......................................................................................................... cudaEventQuery ............................................................................................................ cudaEventSynchronize .................................................................................................... cudaEventDestroy .......................................................................................................... cudaEventElapsedTime ................................................................................................... Error Handling ..................................................................................................................... cudaGetLastError .......................................................................................................... cudaGetErrorString ........................................................................................................

33 33 33 33 34 34 34 34 34 35 35 35 35 35 36 36 36 36 36 36 37 37 37 37 37 38 38 38 38 39 39 39 39 39 39 39 40 40 40 40 40 40 40 41 41 41 v

....... 41 5. 43 Source Code Listing ................................................................................................... 41 cudaRuntimeGetVersion ............... 47 vi ...................... 41 cudaDriverGetVersion .............................................................................................................................................................................................................................................................. 43 Source Code Description .......................... Examples ......................................................................... 43 Matrix Multiplication Example ................. Contact Information ..............................Version Management ................................................................................................................................. 45 6...............................................................................................................................................................

........................ Arithmetic and Bitwise Atomic Functions ...................................................................................................Tables 2................... Fortran Bit Manipulation Intrinsics ....................................................6................. 6 3............. Fortran Reduction Intrinsics .................9.. Fortran Numeric Inquiry Intrinsics ........................... 29 vii ....... 23 3...................... 24 3.................5.......... 23 3......................................... Device Code Intrinsic Datatypes ........7............................... 21 3............................................................... 27 3..................................................................................................10...2.......... Compare and Swap Atomic Function ............ 24 3....1...................................................8............ 22 3... Intrinsic Datatypes ........................................................3............................ 23 3.......................................4.................................... 28 3................................................. Fortran Mathematical Intrinsics .......... Counting Atomic Functions ............................... Fortran Numeric and Logical Intrinsics .................................................................................1.......................................................... Fortran Random Number Intrinsics ..................

viii .

and/or C++ languages. “Runtime APIs” describes the interface between CUDA Fortran and the CUDA Runtime API Chapter 5. The PGI tools are available on a variety of operating systems for the X86. and Intel 64 hardware platforms. Intended Audience This guide is intended for application programmers.Preface This document describes CUDA Fortran. “Programming Guide” serves as a programming guide for CUDA Fortran Chapter 3. “Reference” describes the CUDA Fortran language reference Chapter 4. Organization The organization of this document is as follows: Chapter 1. “Examples” provides sample code and an explanation of the simple example. ix . This guide assumes familiarity with basic operating system usage. a small set of extensions to Fortran that supports and is built upon the CUDA computing architecture. C. AMD64. Conventions This guide uses the following conventions: italic is used for emphasis. “Introduction” contains a general introduction Chapter 2. scientists and engineers proficient in programming with the Fortran.

filename . x . Geneva. ellipsis indicate a repetition. 1997 (Fortran 95).com/cuda. PGI provides a glossary of terms which you can access at www. There are a wide variety of releases and distributions of each of these types of operating systems.Terminology Constant Width is used for filenames. 7/21/2010. you must select either item2 or item3. [ item1 ] in general. examples.pdf. directories.nvidia.. including assembly language statements. NVIDIA. In this case.com/support/definitions. In this example. FORTRAN Fortran language statements are shown in the text of this guide using a reduced fixed point size.com/doc/pgiug. and for language statements in the text.1. Release 10. • NVIDIA CUDA Programming Guide. arguments.nvidia. NVIDIA. • NVIDIA CUDA Compute Unified Device Architecture Reference Manual. Terminology If there are terms in this guide with which you are unfamiliar.com/cuda..pgroup. June 2010. Available online at http:// www. { item2 | item 3} braces indicate that a selection is required. Version 3. Available online at http:// www. In this case item1 is optional.1. Information Technology – Programming Languages – FORTRAN. C/C++ C/C++ language statements are shown in the test of this guide using a reduced fixed point size. The PGI compilers and tools are supported on both 32-bit and 64-bit variants of the Linux and Windows operating systems on a variety of x86-compatible processors. Version 3. square brackets are required to specify a p/t-set. The Portland Group. • PGI Users Guide. September. Zero or more of the preceding item may occur.htm Related Publications The following documents contain additional information related to CUDA Fortran programming.1. square brackets indicate optional items. Bold is used for commands.pgroup.9. options. • ISO/IEC 1539-1:1997. 2009. multiple filenames are allowed. Available online at http://www. In the context of p/tsets.

highly parallel computational units with very high memory bandwidth. PGI 2011 includes support for CUDA Fortran on Linux. compute-intensive programs. CUDA Fortran is a lower-level explicit programming model with substantial runtime library components that give expert programmers direct control of all aspects of GPGPU programming.Chapter 1. shared memory and barrier synchronization within thread groups. The CUDA Fortran extensions described in this document allow the following operations in a Fortran program: • Declaring variables that are allocated in the GPU device memory • Allocating dynamic memory in the GPU device memory • Copying data from the host memory to the GPU memory. CUDA comes with an extended C compiler. A CUDA programmer must partition the program into coarse grain blocks that can be executed in parallel. A properly designed CUDA program will run on any CUDA-enabled GPU. CUDA Fortran includes a Fortran 2003 compiler and tool chain for programming NVIDIA GPUs using Fortran. GPU designs are optimized for the computations found in graphics rendering. CUDA Fortran is an analog to NVIDIA's CUDA C compiler. Mac OS X and Windows. which can cooperate using shared memory and barrier synchronization. regardless of the number of available processor cores. and back • Writing subroutines and functions to execute on the GPU • Invoking GPU subroutines from the host • Allocating pinned memory on the host • Using asynchronous transfers between the host and GPU 1 . a small set of extensions to Fortran that supports and is built upon the CUDA computing architecture. The programming model supports four key abstractions: cooperating threads organized into thread groups. Each block is partitioned into fine grain threads. Graphic processing units or GPUs have evolved into programmable. Introduction Welcome to Release 2011 of PGI CUDA Fortran. here called CUDA C. and tremendous potential for many applications. NVIDIA introduced CUDA™. allowing direct programming of the GPU from a high level language. with compilers and libraries to support the programming of NVIDIA GPUs. and coordinated independent thread groups organized into a grid. Compared to the PGI Accelerator directives-based model and compilers. but are general enough to be useful in many data-parallel. a general purpose parallel programming architecture.

2 .

each instance will be executed by a different CUDA thread. a kernel is called using special chevron syntax to specify the number of thread blocks and threads within each thread block: ! Kernel definition attributes(global) subroutine ksaxpy( n. Such a subroutine is called a device kernel or kernel. y ) real. each thread performs one iteration of the common SAXPY loop operation. Programming Guide This chapter introduces the CUDA programming model through examples written in CUDA Fortran. 3 . i i = (blockidx%x-1) * blockdim%x + threadidx%x if( i <= n ) y(i) = a * x(i) + y(i) end subroutine ! Host subroutine subroutine solve( n. A kernel is defined using the attributes(global) specifier on the subroutine statement. a. In this example. Each thread is assigned a thread block index accessed through the built-in blockidx variable. each with 64 threads. a. dimension(*) :: x. and a thread index accessed through threadidx. the call to the kernel ksaxpy specifies n/64 thread blocks. and each thread has a global thread block index. value :: a integer. “Reference. value :: n. x. y real :: a integer :: n ! call the kernel call ksaxpy<<<n/64. refer to Chapter 3. For a reference for CUDA Fortran.Chapter 2. x. y ) end subroutine In this case. A call to a kernel specifies how many parallel instances of the kernel must be executed. dimension(*) :: x. 64>>>( n. CUDA Fortran Kernels CUDA Fortran allows the definition of Fortran subroutines that execute in parallel on the GPU when called from the Fortran program which has been invoked and is running on the host.” on page 13. x. The CUDA threads are organized into thread blocks. device. a.y real. y ) real. and a local thread index within its thread block.

4 . computed from the thread index. Each thread can also have private local memory. In CUDA Fortran. For a one-dimensional thread block. the host program can directly access data in the host main memory. The shared memory acts like a low-latency. The thread blocks are organized into a one. the first argument in the chevron <<<>>> syntax is the grid size. When invoking a kernel. each with the same thread block size. data in thread local memory may be implemented as processor registers or may be allocated in the global device memory. The thread index may be a one-. so each thread has a thread index within the block. For a two-dimensional thread block of size (Dx. high bandwidth software managed cache memory. Threads in the same thread block can access and share data in shared memory. whether it is a kernel.or two-dimensional grid of blocks. designating whether it is executed on the host or on the device. and if the latter. called from the host. two-. Currently. On the device side. Accesses to constant memory are typically faster than accesses to global memory. is called a host subprogram. • A subprogram declared with attributes(host). the thread ID is equal to (x+Dx(y-1)). again implemented using DMA access. The host can also set the values in the device constant memory. the thread index for each dimension starts at one. all threads can read data in constant memory. A unique thread ID is assigned to each thread. the thread index is equal to the thread ID. such data copies require DMA access to the device. • A subprogram declared with attributes(global) or attributes(device) is called a device subprogram. best performance will often be obtained when thread local data Subroutine / Function Qualifiers A subroutine or function in CUDA Fortran has an additional attribute. the maximum number of threads in a thread block is 512. but it is read-only to the threads and limited in size. or called from another device subprogram. Thread blocks must be able to execute independently.Dy).Dz).Dy. by the same core or by different cores. A kernel may be invoked with many thread blocks. or with the host attribute by default. the thread ID is (x+Dx(y-1)+Dy(z-1)). Each thread in the block waits at the call to SYNCTHREADS() until all threads have reached that call. On the host side. and the second argument is the thread block size. Threads in the same thread block may cooperate by using shared memory. and by synchronizing at a barrier using the SYNCTHREADS() intrinsic.Thread Blocks Thread Blocks Each thread is assigned a thread block index accessed through the built-in blockidx variable. data in shared memory has a lifetime of the thread block. and a block index within the grid. data in global device memory can be read or written by all threads. Memory Hierarchy CUDA Fortran programs have access to several memory spaces. and a thread index accessed through threadidx. two thread blocks may be executed in parallel or one after the other. It can also directly copy data to and from the device global memory. or three-dimensional index. so are slow relative to the host memory. Data in constant memory space is initialized by the host program. For a three-dimensional thread block of size (Dx.

5 . The default is attributes(host). and is allocated in the device global memory.” on page 7. specified on the subroutine or function statement. declares that the subprogram is to be executed on the device. Dummy arguments in a device subprogram must not have the pointer attribute. • If declared in a module. • A device subprogram must not have optional arguments. shared. or device attributes is specified. Variable Qualifiers Variables in CUDA Fortran have a new attribute. such a routine must be called from a subprogram with the global or device attribute. At most one of the device. Attributes(host) The host attribute. • A device subprogram must not contain variables with the SAVE attribute. Programming Guide • A subroutine declared with attributes(global) is also called a kernel subroutine. Such a call is asynchronous. Restrictions The following restrictions apply to subprograms: • A device subprogram must not be recursive. the variable may be accessed by any subprogram in that module and by any subprogram that uses the module. global. constant. specified on the subroutine or function statement. it declares that the subroutine is a kernel subroutine. declares that the subroutine or function is to be executed on the host. • A kernel subroutine may not also have the device or host attribute. or with data initialization. the host routine making the call continues to execute before the device has completed its execution of the kernel subroutine. Attributes(device) The device attribute. • Calls to a kernel subroutine must specify the execution configuration. if none of the host. to be executed on the device. Such a subprogram can only be called from another host subprogram. By default. and may only be called from the host using a kernel call containing the chevron syntax and runtime mapping parameters. which declares in which memory the data is allocated. or pinned attributes may be specified for a variable. and may not contain any subroutines or functions. variables declared in modules or host subprograms are allocated in the host main memory. Attributes(global) The global attribute may only be specified on a subroutine statement. that is.Chapter 2. as described in “Predefined Variables in Device Subprograms. • Device subprograms may not be contained in a host subroutine or function. Attributes(device) A variable with the device attribute is called a device variable.

2. An allocatable device variable has a dynamic lifetime. Datatypes in Device Subprograms The following intrinsic datatypes are allowed in device subprograms and device data: Table 2. but may be modified in host subprograms. A shared variable may only be declared in a device subprogram. A shared variable may not be data initialized. When a pinned variable is allocated. Device constant variables are allocated in the device constant memory space. an allocatable array.8 1. Some operating systems or installations may restrict the use.1. Intrinsic Datatypes Type integer logical real double precision Type Kind 1. though a write in one thread is only guaranteed to be visible to other threads after the next call to the SYNCTHREADS() intrinsic. availability. the variable will be allocated in the normal host paged memory.8 4.8 equivalent to real(kind=8) 6 . or size of page-locked memory. It can be read or written by all threads in the block. or an assumed-shape dummy array.2. from when it is allocated until it is deallocated. it will be allocated in host pagelocked memory. and has a lifetime of the thread block. and may only be accessed within that subprogram. or by other device subprograms to which it is passed as an argument. the variable may be accessed by that subprogram or subprograms contained in that subprogram. Device constant data may not be assigned or modified in any device subprogram. and have a lifetime of the entire application. Attributes(constant) A variable with the constant attributes is called a device constant variable.4.Datatypes in Device Subprograms • If declared in a host subprogram. Other device variables have a lifetime of the entire application. Device constant variables may not be allocatable. The advantage of using pinned variables is that copies from page-locked memory to device memory are faster than copies from normal paged host memory. the variable may be accessed by any subprogram in that module and by any subprogram that uses the module. if the allocation in page-locked memory fails. A pinned variable must be an allocatable array. If declared in a module. A device array may be an explicit-shape array. A shared variable is allocated in the device shared memory for a thread block. Attributes(pinned) A variable with the pinned attributes is called a pinned variable. Attributes(shared) A variable with the shared attributed is called a device shared variable or a shared variable.4.

the value of griddim%z is always one. These variables are of type dim3. or are type(dim3). in addition to the statically allocated shared memory.…) where • grid and block are either integer expressions (for one-dimensional grids and thread blocks). called warps. the module cudafor defines the derived type dim3 as follows: type(dim3) integer(kind=4) :: x. warpsize contains the number of threads in a warp.streamid]]>>>(arg1.Chapter 2. This memory is used for the 7 .8 1 Derived types may contain members with these intrinsic datatypes or other allowed derived types. • The variable blockdim contains the dimensions of the thread block.z end type These predefined variables. it specifies the number of bytes of shared memory to be allocated for each thread block. • The value of bytes must be an integer. the value of grid%z must be one. blockdim has the same value for all thread blocks in the same grid. in bytes. Predefined Variables in Device Subprograms Device subprograms have access to block and grid indices and dimensions through several builtin read-only variables. and a stream identifier. the threadidx%y and/or threadidx%z components have the value one.or two-dimensional grids and one-. and block%x and block%y must be equal to or greater than one. except for warpsize. Programming Guide complex character(len=1) 4. • The variable is warpsize is declared to be type integer. for one. The execution configuration defines the dimensionality and extent of the grid and thread blocks that execute the subroutine. • If block is type(dim3). the value of each component must be equal to or greater than one. • The variable griddim contains the dimensions of the grid. • The variable blockidx contains the block index within the grid.block[. are not accessible in host subprograms. blockidx%y has the value one. The value of blockidx%z is always one. to support concurrent stream execution on the device.arg2. • If grid is type(dim3).y. It may also specify a dynamic shared memory extent.bytes[. Threads are executed in groups of 32. for one-dimensional grids. Execution Configuration A call to a kernel subroutine must specify an execution configuration. for one. and the product of the component values must be less than or equal to 512. two-. as with threadidx. • The variable threadidx contains the thread index within its thread block. A kernel subroutine call looks like this: call kernel<<<grid.or two-dimensional thread blocks. or three-dimensional thread blocks.

By calling the runtime routine cudaThreadSynchronize. the call actually returns to the host program before the kernel subroutine begins execution. inserting synchronization into the kernel as appropriate. the host program can synchronize and wait for all previously launched or queued kernels. stream zero is special in that operations on the stream zero queue will begin only after all preceding operations on all queues are complete. Each user-created stream manages its own queue. including calling or queueing more kernels for execution on the device. Kernel Loop Directive CUDA Fortran allows automatic kernel generation and invocation from a region of host code containing one or more tightly nested loops. it specifies the stream to which this call is associated. used when no stream identifier is specified. The one exception to this rule is a scalar reduction operation. Concurrent Host and Device Execution When a host subprogram calls a kernel subroutine. the compiler handles the generation of the final reduction kernel. its value is treated as zero. the host can continue executing. see Section 3. Concurrent Stream Execution Operations involving the device. and will only be initiated when all previous operations on that queue have been completed. Asynchronous concurrent execution There are two components to asynchronous concurrent execution with CUDA Fortran. As with any kernel. such as summing the values in a vector or matrix. An application can manage more concurrency by using multiple streams. and may execute concurrently with each other. The work in the loops specified by the directive is executed in parallel. operations on different stream queues may execute out-of-order with respect to when they were placed on the queues. the host program should synchronize with the device to avoid erroneous results. in cases where the host program reads or modifies device or constant data. are implemented using stream queues. 8 . including kernel execution and data copies to and from device memory.Asynchronous concurrent execution assumed-size shared variables in the thread block. where the launch actually corresponds to placing the kernel on a queue for execution by the device. the launch is asynchronous.3.2. • The value of streamid must be an integer greater than or equal to zero. An operation is placed at the end of the stream queue. and no subsequent operations on any queue begin until the stream zero operation is complete. is stream zero. Programmers must be careful when using concurrent host and device execution. For these operations. Launch configuration and mapping of the loop iterations onto the hardware is controlled and specified as part of the directive body using the familiar CUDA chevron syntax. it is the programmer's responsibility to ensure that parallel execution is legal and produces the correct answer. across the thread blocks and grid. The call can be treated as a kernel launch operation. The default stream. If not specified. In this way. The program can use cudaThreadSynchronize() or CUDA Events to wait for the completion of the kernel.

starting at loop n and working out.*). !$cuf kernel do(2) <<< *. using asterisks tells the compiler to choose a thread block shape and/or compute the grid shape from the thread block shape and the loop limits. that is. Alternatively. m do i = 1. The grid shape.Chapter 2. the directive defines a two-dimensional thread block of size 32x4.j) = b(i.j) end do end do 9 . The inner i dimension is run sequentially on each thread. block >>> The compiler maps the launch configuration specified by the grid and block values onto the outermost n loops.4) >>> do j = 1. only the outer j loop is run in parallel with a thread block size of 256.j) end do end do In this example. ThreadIdx%y runs from 1 to 4 and is mapped onto the outer j loop.j) end do end do Without an explicit n on the do.j) +c(i.j) = b(i. the schedule applies just to the outermost loop. You might consider if the code in Example 2 would perform better if the two loops were interchanged. is computed by the compiler and runtime by dividing the loop trip counts n and m by the thread block size.*).j) +c(i. Alternatively. n a(i.j) = b(i.1) >>> do j = 1. 256 >>> do j = 1. m do i = 1. (256. (32. you could specify a configuration like the following in which the threads read and write the matrices in coalesced fashion.j) +c(i. the default value is 1. The grid and block values can be an integer scalar or a parenthesized list. The body of the doubly-nested loop is turned in the kernel body: ThreadIdx%x runs from 1 to 32 and is mapped onto the inner i loop. Example 1 !$cuf kernel do(2) <<< (*. n a(i. Loops which are not mapped onto the grid and block values are run sequentially on each thread. so all iterations are computed. Programming Guide Syntax The general form of the kernel directive is: !$cuf kernel do[(n)] <<< grid. specified as (*. In this case. n a(i. Example 2 !$cuf kernel do <<< *. m do i = 1.

but the device-specific intrinsics such as syncthreads. and increment must be invariant with respect to any other loop in the kernel do. as illustrated here: !$cuf kernel do(2) <<< (1. In CUDA built-in language. one for each thread. • Scalars used or assigned in the loop must either have the device attribute. Specifically. incrementing by 256. • There can be no GOTO or EXIT statements within or between any loops that have been mapped onto the grid and block configuration values. Further expansion of the work along the i direction and all work across the j dimension is handled by the mapping onto the grid dimensions. etc.j) +c(i. One thread block is created for each j.*). the 256 threads in each block each do one element of the matrix addition. To "unroll" more work into each thread.Kernel Loop Directive Example 3 In Example 2. m do i = 1. • CUDA Fortran intrinsic functions are allowed. because the compiler skips over the constant 1 in the i loop grid size. loops. or the compiler will make a device copy of that variable live for the duration of the loops. n a(i. gridDim%x is equal to m. in concert. Restrictions on the CUF kernel directive The following restrictions apply to CUF kernel directives: • If the directive specifies n dimensions.1) >>> do j = 1. IF statements. atomic functions. • Only CUDA Fortran dataypes are allowed within the loops. the compiler generates a correct sequence of synchronized operations to produce one copy either in device global memory or on the host. and GOTO statements. Summation Example The simplest directive form for performing a dot product on two device arrays takes advantage of the properties for scalar use outlined previously. 10 . • Implicit loops and F90 array syntax are not allowed within the directive loops. it must be followed by at least that many tightly-nested DO loops. • Arrays used or assigned in the loop must have the device attribute. • Subroutine and function calls to attributes(device) subprograms are allowed if they are in the same module as the code containing the directive. (256. Except in the case of reductions. if they are allowed in device code. specify non-asterisk values for the grid. • The tightly-nested DO loops must have invariant loop limits: the lower limit. the j loop is mapped onto the grid x-dimension. • The body of the loops may contain assignment statements. upper limit.j) = b(i.j) end do end do Now the threads in a thread block handle all values in the i direction. are not. when a reduction has a scalar target.

This CUF kernel can be followed by another CUF kernel in the same subprogram: !cuf kernel do <<< *. the host floating point units and intrinsics are used. To build a program using emulation mode. The . emulation mode may execute a single thread block at a time.0 !cuf kernel do <<< *. compile and link with the –Mcuda=emu command line option. the compiler recognizes the use of the scalar and generates just one final result. In emulation mode. * >>> do i = 1. In emulation mode. use of rsum on the host following this loop is undefined. such as memory races. * >>> do i = 1. n rsum = rsum + x(i) * y(i) end do For reductions. It’s important to note that the emulation is far from exact. 11 . This will not expose certain errors. the compiler recognizes rsum as a scalar temporary which should be allocated locally on every thread. To compile a fixed-format program. in which case the program is processed by the preprocessor before being compiled. However. CUDA Fortran extensions can be enabled in any Fortran source file by adding the –Mcuda command line option. the . Emulation Mode PGI Fortran compilers support an emulation mode for program development on workstations or systems without a CUDA-enabled GPU and for debugging.Chapter 2. which may produce slightly different answers than the device units and intrinsics.CUF extension may also be used.cuf extension specifies that the file is a free-format CUDA Fortran program. In particular. n rsum = x(i) * y(i) z(i) = rsum end do In this CUF kernel. add the command line option –Mfixed. the device code is compiled for and runs on the host. Building a CUDA Fortran Program CUDA Fortran is supported by the PGI Fortran compilers when the filename uses a CUDA Fortran extension. allowing the programmer to use a host debugger. Programming Guide rsum = 0.

12 .

as described in the following sections. global. This chapter describes how to specify the new attributes. Global Subroutines The global attribute may be explicitly specified on the Subroutine statement as follows: attributes(global) subroutine sub(…) Functions may not have the global attribute. or device attribute. New Subroutine and Function Attributes CUDA Fortran adds new attributes to subroutines and functions. A subprogram with no attributes prefix has the host attribute by default. their meaning and restrictions. or may have both host and device attribute. so no other subroutine prefixes are allowed.Chapter 3. or a host subprogram. Host Subroutines and Functions The host attribute may be explicitly specified on the Subroutine or Function statement as follows: attributes(host) subroutine sub(…) attributes(host) integer function func(…) integer attributes(host) function func(…) The host attributes prefix may be preceded or followed by any other allowable subroutine or function prefix specifiers (recursive. if there is no attributes prefix on the subprogram statement. or both. A kernel subroutine may not be recursive. and may not contain any other subprogram. A Subroutine may have the host. These attributes are specified using the attributes(attr) prefix on the Subroutine or Function statement. A Function may have the host or device attribute. A kernel subroutine may not be contained in another subroutine or function. or elemental. A kernel subroutine is compiled as a kernel for execution on the device. to be called from a host routine using an execution configuration. Reference This chapter is the CUDA Fortran Language Reference. A subroutine with the global attribute is called a kernel subroutine. then default rules are used. A host subprogram is compiled for execution on the host processor. elemental. pure. function return datatype). pure. 13 . A subroutine or function with the host attribute is called a host subroutine or function.

14 .Variable Attributes Device Subroutines and Functions The device attribute may be explicitly specified on the Subroutine or Function statement as follows: attributes(device) subroutine sub(…) attributes(device) datatype function func(…) datatype attributes(device) function func(…) A subroutine or function with the device attribute may not be recursive. a and b. A subroutine or function with the device or kernel attribute is called a device subprogram. the array may be declared to be in pinned memory. • It may not be contained in another subroutine or function. • It may not be pure or elemental. except for the function return datatype. The following example declares two arrays. that is.” on page 21. or elemental. nor have the pure or elemental prefix on the subprogram statement. in the thread block shared memory. in constant memory space. nor have the recursive prefix on the subprogram statement. The device attribute can be specified with the attributes statement. or as an attribute on the type declaration statement. pure. Variable Attributes CUDA Fortran adds new attributes for variables and arrays. to be device arrays of size 100. so no other subroutine or function prefixes are allowed. The advantage of using pinned memory is that transfers between the device and pinned memory are faster and can be asynchronous. Device data A variable or array with the device attribute is defined to reside in the device global memory. Variables in modules may be declared to be in device global memory or constant memory space. Variables declared in a device subprogram may have one of four attributes: they may be declared to be in device global memory. • It may not contain another subprogram. Restrictions on Device Subprograms A subroutine or function with the device or global attribute must satisfy the following restrictions: • It may not be recursive. CUDA Fortran adds a new attribute for allocatable arrays in host memory. For more information. or in thread local memory. and may only be called from device subprograms in the same module. This section describes how to specify the new attributes and their meaning and restriction. A device subprogram is compiled for execution on the device. refer to “Device code. A subroutine or function with the device attribute must appear within a Fortran module. in page-locked host memory space.

in that case. • Device variables and arrays declared in a host subprogram cannot have the Save attribute. Constant data A variable or array with the constant attribute is defined to reside in the device constant memory space. • Device variables and arrays may be passed as actual arguments to host and device subprograms. c and d. Target. In host subprograms. real :: c(100) attributes(constant) :: c real. • Device variables and arrays may appear in modules. constant :: d(100) These rules apply to constant data: • Constant data may not have the Pointer. but may not be in a Common block or an Equivalence statement. the subprogram interface must be explicit (in the Fortran sense). to be constant arrays of size 100. • Constant variables and arrays may appear in modules. or as an attribute on the type declaration statement. but may not be in a Common block or an Equivalence statement. The constant attribute can be specified with the attributes statement.Chapter 3. • Members of a derived type may not have the device attribute unless they are allocatable. • Device variables and arrays may not have the Pointer or Target attributes. device :: b(100) These rules apply to device data: • An allocatable device array dynamically allocates device global memory. The following example declares two arrays. or Allocatable attributes. device data may only be used in the following manner: • In declaration statements • In Allocate and Deallocate statements • As an argument to the Allocated intrinsic function • As the source or destination in a data transfer assignment statement • As an actual argument to a kernel subroutine • As an actual argument to another host subprogram or runtime API call • As a dummy argument in a host subprogram A device array may have the allocatable attribute. 15 . or may have adjustable extent. and the matching dummy argument must also have the device attribute. Reference real :: a(100) attributes(device) :: a real.

shared :: x(*) Such an array has special significance. • Within host subprograms. variables and arrays with the constant attribute may be read and written. In host subprograms. as long as the interface is explicit and the matching dummy argument has the shared attribute. or as an attribute on the type declaration statement. A shared variable or array may only be declared and used inside a device subprogram. • Constant variables and arrays may be passed as actual arguments to host and device subprograms. to be shared arrays of size 100.Variable Attributes • Members of a derived type may not have the constant attribute. • Arrays with the constant attribute must have fixed size. The shared attribute can be specified with the attributes statement. • Shared variables and arrays may be passed as actual arguments to from a device subprogram to another device subprogram. • Within device subprograms. the last dimension of a shared array may have an asterisk as its upper bound: real. • Members of a derived type may not have the shared attribute. variables and arrays with the constant attribute may not be assigned or modified. Shared arrays that are not dummy arguments may be declared as assumed-size arrays. Its size is determined at run time by the call to the kernel. and the matching dummy argument also has the constant attribute. the value of the bytes argument in the execution configuration is used to specify the number of bytes of shared memory that is dynamically allocated for each thread block. s and t. real :: c(100) attributes(shared) :: c real. or Allocatable attributes. The following example declares two arrays. that is. Target. This memory is used for the 16 . • Shared variables may not be in a Common block or Equivalence statement. as long as the subprogram interface is explicit. When the kernel is called. shared :: d(100) These rules apply to shared data: • Shared data may not have the Pointer. data with the constant attribute may only be used in the following manner: • In declaration statements • As the source or destination in a data transfer assignment statement • As an actual argument to another host subprogram • As a dummy argument in a host subprogram Shared data A variable or array with the shared attribute is defined to reside in the shared memory space of a thread block.

This means the actual argument must be stored in device global memory. and dynamically deallocated using the Deallocate statement. it will be automatically deallocated when the subprogram returns. pinned :: q(:) Pinned arrays may be passed as arguments to host subprograms regardless of whether the interface is explicit. The pinned attribute can be specified with the attributes statement.Chapter 3. or as an attribute on the type declaration statement. The following example declares two arrays. dimension(n. the value of n can be passed from the host without needing to reside in device memory. b integer. Pinned arrays An allocatable array with the pinned attribute will be allocated in special page-locked host memory. value :: n In this case. when such memory is available. to be pinned allocatable arrays. Scalar arguments can be passed by value. by adding the value attribute to the variable declaration. specifically for dynamically allocating device arrays and host pinned arrays. n ) real. or whether the dummy argument has the pinned and allocatable attributes. 17 . dummy arguments are passed by default by reference. If a shared array is not a dummy argument and not assumed-size. they are all implicitly equivalenced. Programmers must take this into account when coding. Reference assumed-size shared memory arrays in that thread block. or the deallocation may fail. p and q. If a device array declared in a host subprogram does not have the Save attribute.n) :: a. Allocating Device and Pinned Arrays This section describes extensions to the Allocate statement. Allocating Device Memory Device arrays can have the allocatable attribute. and the address of the argument is passed to the subprogram. Value dummy arguments In device subprograms. These arrays are dynamically allocated in host subprograms using the Allocate statement. The variable arrays corresponding to the dummy arguments a and b must be set up before the call to reside on the device. if there is more than one assumed-size shared memory array. allocatable. following the rules of Fortran. Where the array is deallocated. the declaration for the array must still have the pinned attribute. real :: p(:) allocatable :: p attributes(pinned) :: p real. attributes(global) subroutine madd( a. An array with the pinned attribute may be declared in a module or in a host subprogram. b. starting at the same shared memory address. and other supported methods for allocating device memory. it must be fixed size. as is done in C.

Allocating Pinned Memory Allocatable arrays with the pinned attribute are dynamically allocated using the Allocate statement. allocatable. device :: atmp(4) … end subroutine vfunc ! adev and atmp are deallocated Allocating Device Memory Using Runtime Routines For programmers comfortable with the CUDA C programming environment. the compiler allocates the array in normal paged host memory.Allocating Device and Pinned Arrays real. These functions return memory which will bypass certain Fortran allocatable properties such as automatic deallocation. allocatable. allocatable. refer to“Memory Management. device :: v(:) istat = cudaMalloc(v. Mixing standard Fortran allocate/deallocate with the runtime Malloc/Free for a given array is not supported.” on page 32. Otherwise. These arrays will also have a lifetime of the subprogram as is usual with the Fortran language: subroutine vfunc(a. pinned :: p(:) allocate(p(5000). Fortran interfaces to the CUDA memory management runtime routines are provided. The cudaMalloc function can be used to allocate single-dimensional arrays of the supported intrinsic datatypes.stat=istat) … if(allocated(b)) deallocate(b) Scalar variables can be allocated on the device using the Fortran 2003 allocatable scalar feature. To use these. 100) … istat = cudaFree(v) For a complete list of the memory management runtime routines. real. allocatable.c. If no such memory space is available. or if it is exhausted. and cudaFree can be used to free it: real.stat=istat) … if(allocated(p)) deallocate(p) 18 .n) real. device :: b(:) allocate(b(5024). if available. declare and initialize the scalar on the host as: integer. device :: ndev allocate(ndev) ndev = 100 The language also supports the ability to create the equivalent of automatic and local device arrays without using the allocate statement. The compiler will generate code to allocate the array in host page-locked memory. pinned allocatable arrays work and act like any other allocatable array on the host. and thus the arrays are treated more like C malloc’ed areas. device :: adev(n) real.

Data transfer between host and device memory This section provides methods to transfer data between the host and device memory.not. Compiler-generated temporary arrays are generated to accommodate the host copies of device data as needed. which normally only allows one device array to appear on the righthand-side of an expression. In general. plog) then . and cdev are conforming device arrays. the following expressions are legal: a = adev adev = a b = a + adev c = x * adev + b 19 . bdev. . the rule of thumb is all arithmetic or operations must occur on the host. b. Implicit Data Transfer in Expressions Some limited data transfer can be enclosed within expressions. Similarly. and the right hand is a host variable or host array or array section. This means such data copies are synchronous. For instance. copies data from the device global memory to the host memory. and the right hand side is a device variable or device array or array section.Chapter 3. It returns a logical success value. pinned=plog) if (. stat=istat. Note Using assignment statements to read or write device or constant data implicitly uses CUDA stream zero. . you can use simple assignment statements to copy or assign variables or arrays with the constant attribute. an additional PINNED keyword is added to the allocate statement. • An assignment statement where the left hand side is a device variable or device array or array section. logical plog allocate(p(5000). Data Transfer Using Assignment Statements You can copy variables and arrays from the host memory to the device memory by using simple assignment statements in host subprograms. copies data from the host memory to the device global memory. meaning the data copy waits until all previous kernels and data copies complete. and adev. Reference To determine whether or not the allocation from page-locked memory was successful. • An assignment statement where the left hand side is a host variable or host array or array section. if a. • An assignment statement with a device variable or device array or array section on both sides of the assignment statement copies data between two device variables or arrays. and c are conforming host arrays.

block. amount of contiguous data in the slices. The execution configuration is specified after the subroutine name in the call statement. Counts expressed in arguments to the Fortran runtime routines are expressed in terms of data type elements. Data Transfer Using Runtime Routines For programmers comfortable with the CUDA C programming environment. Invoking a kernel subroutine A call to a kernel subroutine must give the execution configuration for the call. device to host. it is converted to dim3(block. The execution configuration gives the size and shape of the grid and thread blocks that execute the function. Array slices are also supported. Fortran interfaces to the CUDA memory management runtime routines are provided. 20 . If it is type(dim3). If block is an integer. device :: wrk(1024) real cur(512) istat = cudaMemcpy(wrk.1). the kind parameter to the Memcpy routines is optional in Fortran since the attributes of the arrays are explicitly declared. The product grid%x*grid%y gives the number of thread blocks to launch. stream >>> • grid is an integer. If it is type(dim3). The cudaMemcpy function can be used to copy data between the host and the GPU: real. 512) For those familiar with the CUDA C routines. as well as the amount of shared memory to use for assumed-size shared memory arrays. bl • block is an integer. and the implementation. or of type(dim3). These functions can transfer data either from the host to device. and their performance is dependent on the size of the slice.” on page 32. the number of threads per thread block is block%x*block%y*block%z. refer to “Memory Management. the value of grid%z must be one.Invoking a kernel subroutine The following expressions are not legal as they either promote a false impression of where the actual computation occurs. or of type(dim3). it has the form: <<< grid. For a complete list of memory management runtime routines. which must be less than the maximum supported by the device. If grid is an integer. cur. or both: c = adev + bdev adev = adev + a b = sqrt(adev) Elemental transfers are supported by the language but will perform poorly. or would be more efficient written in another way.1. or from one device array to another. not bytes.1). bytes. it is converted to dim3(grid. and the associated stream.1.

The system module cudafor includes definitions of the derived type dim3.8 4(default).5 on page 41.y. or declared in device subprograms.z end type 21 .1.8 1 (default) Additionally. it must be an integer. if present. • stream is optional. constant. if present. they may be of derived type. or if bytes is greater than the shared memory available.” on page 16. It specifies the stream to which this call is enqueued.2.2. Reference • bytes is optional. or another allowed derived type. where the members of the derived type have one of the allowed intrinsic datatypes. DB >>> ( A ) The function call will fail if the grid or block arguments are greater than the maximum sizes allowed. allowing for static shared memory declared in the kernel and for other dedicated uses.4(default). refer to “Shared data. For more information. Device code Datatypes allowed Variables and arrays with the device. and specifies the number of bytes of shared memory to be allocated for each thread block to use for assumed-size shared memory arrays.8 equivalent to real(kind=8) 4(default). such as the function arguments and execution configuration arguments. or a value returned by a call to cudaStreamCreate. or shared attributes. For instance.Chapter 3. They may have any of the intrinsic datatypes in the following table. defined as type(dim3) integer(kind=4) :: x. and have a value of zero. are limited to the types described in this section.8 1. If not specified. See Section 4. it must be a scalar integer. the value zero is used. a kernel subroutine attributes(global) subroutine sub( a ) can be called like: call sub <<< DG. Device Code Intrinsic Datatypes Type integer logical real double precision complex character(len=1) Type Kind 1.4(default). Table 3.

real. • The variable blockdim contains the dimensions of the thread block. the value of griddim%z is always one.or two-dimensional thread blocks. blockidx. blockdim has the same value for all threads in the same grid. The value of griddim is the same for all threads in the same grid. real 22 .2. the blockdim%y and/or blockdim %z components have the value one. The value of blockidx is the same for all threads in the same thread block. the threadidx%y and/or threadidx%z components have the value one. the value of griddim%y is one for one-dimensional grids.Device code Builtin variables The system module cudafor declares several predefined variables.real) complex integer. blockidx. and griddim are available only in device subprograms. complex complex real real real real or (real. • The variables threadidx. blockidx%y has the value one. Fortran intrinsics This section lists the Fortran intrinsic functions allowed in device subprograms. for one. real integer. for onedimensional grids. • The variable griddim contains the dimensions of the grid. These variables are read-only. for one. blockdim. • The variable warpsize contains the number of threads in a warp. blockdim. It has constant value. • The variable blockidx contains the block index within the grid. as with threadidx. Fortran Numeric and Logical Intrinsics Table 3. They are declared as follows: type(dim3) :: threadidx. Fortran Numeric and Logical Intrinsics Name abs aimag aint anint ceiling cmplx conjg dim floor Argument Datatypes integer. real real integer. complex integer. The value of blockidx%z is always one. real. real real Name int logical max min mod modulo nint real sign Argument Datatypes integer. griddim integer(4) :: warpsize • The variable threadidx contains the thread index within its thread block. complex logical integer. real integer.or two-dimensional thread blocks. the value of griddim%z is always one. real integer. real. currently defined to be 32.

Chapter 3. complex Name log log10 sin sinh sqrt tan tanh Argument Datatypes real.integer) real Fortran Bit Manipulation Intrinsics Table 3. real. real real integer. complex integer. Reference Fortran Mathematical Intrinsics Table 3.3. Fortran Bit Manipulation Intrinsics Name btest iand ibclr ibits ibset ieor ior Argument Datatypes integer integer integer integer integer integer integer Name ishft ishftc leadz mvbits not popcnt poppar Argument Datatypes integer integer integer integer integer integer integer 23 . real integer. real real real Name precision radix range selected_int_kind selected_real_kind tiny Argument Datatypes real. Fortran Numeric Inquiry Intrinsics Name bit_size digits epsilon huge maxexponent minexponent Argument Datatypes integer integer. complex real real Fortran Numeric Inquiry Intrinsics Table 3. complex real real.real) real.4. Fortran Mathematical Intrinsics Name acos asin atan atan2 cos cosh exp Argument Datatypes real real real (real.5. complex real real. complex real real. complex integer (integer.

complex integer. . real. real integer. complex Fortran Random Number Intrinsics Table 3. Synchronization Functions The synchronization functions control the synchronization of various threads during execution of thread blocks. real integer. or write-after-write hazards for some of these memory accesses. real. write-after-read. Fortran Random Number Intrinsics Name random_number random_seed Argument Datatypes real integer New Intrinsic Functions This section describes the new intrinsic functions and subroutines supported in device subprograms. This 24 . SYNCTHREADS The syncthreads intrinsic subroutine acts as a barrier synchronization for all threads in a single thread block.Device code Fortran Reduction Intrinsics Table 3. real Name minloc minval product sum Argument Datatypes integer.” on page 32. it has no arguments: void syncthreads() Sometimes threads within a block access the same addresses in shared or global memory. To avoid these potential issues.6. thus creating potential read-after-write.7. use syncthreads()to specify synchronization points in the kernel. real integer. refer to “Thread Management. Fortran Reduction Intrinsics Name all any count maxloc maxval Argument Datatypes logical logical logical integer. syncthreads syncthreads_count syncthreads_and syncthread_or threadfence threadfence_block threadfence_system For detailed information on these functions.

each block atomically increments a counter to signal that it is done with computing and storing its partial sum. SYNCTHREADS_COUNT integer syncthreads_count(int_value) syncthreads_count. suppose you use a kernel to compute the sum of an array of N numbers in one call. Each block first sums a subset of the array and stores the result in global memory. SYNCTHREADS_AND integer syncthreads_and(int_value) syncthreads_and. You can use threadfence(). all threads must also reach and execute the same call statement. acts a as a barrier at which all threads in the block must wait before any thread is allowed to proceed. Each thread in a thread block pauses at the syncthreads call until all threads have reached that call. syncthreads_and evaluates the integer argument int_value for all threads of the block and returns non-zero if and only if int_value evaluates to non-zero for all of them. like syncthreads. acts as a barrier at which all threads in the block must wait before any thread is allowed to proceed. If no fence is placed between storing the partial sum and incrementing the counter.Chapter 3. syncthreads_or evaluates the integer argument int_value for all threads of the block and returns non-zero if and only if int_value evaluates to non-zero for any of them. Typically. If any thread in a thread block issues a call to syncthreads. or the kernel fails to complete correctly. THREADFENCE void threadfence() threadfence acts as a memory fence. like syncthreads. SYNCTHREADS_OR integer syncthreads_or(int_value) syncthreads_or. To determine which block is finished last. other threads may see the effects of these memory writes in a different order. acts as a barrier at which all threads in the block must wait before any thread is allowed to proceed. when a thread issues a series of writes to memory in a particular order. creating a wait. Memory Fences In general. other threads may see the effects of these memory writes in a different order. When all blocks are done. the last block done reads each of these partial sums from global memory and sums them to obtain the final result. when a thread issues a series of writes to memory in a particular order. In addition. and threadfence_system() to create a memory fence to enforce ordering. For example. like syncthreads. Reference intrinsic acts as a barrier at which all threads in the block must wait before any thread is allowed to proceed. syncthreads_count evaluates the integer argument int_value for all threads of the block and returns the number of threads for which int_value evaluates to non-zero. In addition. 25 . In addition. Threads within a block cooperate and share data by synchronizing their execution to coordinate memory accesses. the counter might increment before the partial sum is stored. threadfence_block().

true. Warp-Vote Operations Warp-vote operations are only supported by devices with compute capability 1. as described in “cudaGetDeviceProperties.Device code threadfence() is one method to enforce a specific order. THREADFENCE_SYSTEM void threadfence_system() threadfence_system acts as a memory fence.0) ) allneg = .” on page 32. Each of these functions has a single argument. ALLTHREADS The allthreads function is a warp-vote operation with a single scalar logical argument: if( allthreads(a(i)<0. GPU_TIME The gpu_time intrinsic returns the value of the clock cycle counter on the GPU. The clock frequency can be determined by calling cudaGetDeviceProperties.true. All global and shared memory accesses made by the calling thread prior to threadfence() are visible to: • All threads in the thread block for shared memory accesses • All threads in the device for global memory accesses THREADFENCE_BLOCK void threadfence_block() threadfence_block acts as a memory fence.2 and higher. The function allthreads evaluates its argument for all threads in the current warp. for all threads in the warp. The value of the function is . 26 . creating a wait until all global and shared memory accesses made by the calling thread prior to threadfence_block() are visible to all threads in the thread block for all accesses. creating a wait until all global and shared memory accesses made by the calling thread prior to threadfence_system() are visible to: • All threads in the thread block for shared memory accesses • All threads in the device for global memory accesses • Host threads for page-locked host memory accesses threadfence_system() is only supported by devices of compute capability 2.0 or higher.true. only if the value of the argument is . It has a single argument: integer(8) clock call gpu_time(clock) The argument to gpu_time is set to the value of the clock cycle counter.

for all threads in the warp. Arithmetic and Bitwise Atomic Functions These atomic functions read and return the value of the first argument.true. These functions are: Table 3. The value of the function is .1 and higher.false. The function anythread evaluates its argument for all threads in the current warp. Example: if( ballot(int_value) ) allneg = . value ) Additional Atomic Update mem = mem + value mem = mem – value 27 . which must be a variable or array element in shared memory (with the shared attribute) or in device global memory (with the device attribute). depending on the function. The atomic functions return correct values even if multiple threads in the same or different thread blocks try to read and update the same location without any synchronization. only if the value of the argument is . This function is only supported by devices of compute capability 2. Compute capability 1. Atomic functions are only supported by devices with compute capability 1.true.Chapter 3. value ) atomicsub( mem. Reference ANYTHREAD The anythread function is a warp-vote operation with a single scalar logical argument: if( anythread(a(i)<0. BALLOT The ballot function is a warp-vote operation with a single integer argument: unsigned integer ballot(int_value) The function ballot evaluates the argument int_value for all threads of the warp and returns an integer whose Nth bit is set if and only if int_value evaluates to non-zero for the Nth thread of the warp. and store the combined value back to the first argument location. Atomic Functions The atomic functions read and write the value of their first operand. Both arguments must be of type integer(kind=4).0) ) allneg = . They also combine that value with the value of the second argument. mem.0.false.2 or higher is required if the first argument has the shared attribute.8. Note The return value for each of these functions is the first argument. Arithmetic and Bitwise Atomic Functions Function atomicadd( mem.

mem>0) then mem = mem-1 else mem = imax endif atomicdec( mem. value ) mem = max(mem. value ) atomicexch( mem. Note The return value for each of these functions is the first argument. depending on the result of the comparison. value ) atomicmin( mem. These functions are intended to implement circular counters. value ) atomicxor( mem. They also compare the first argument with the second argument. imax ) Additional Atomic Update if (mem<imax) then mem = mem+1 else mem = 0 endif if (mem<imax .value) mem = iand(mem. counting up to or down from a maximum value specified in the second argument.value) mem = min(mem. value ) atomicor( mem.value) mem = value Counting Atomic Functions These atomic functions read and return the value of the first argument. Note The return value for this function is the first argument. All three arguments must be of type integer(kind=4). Both arguments must be of type integer(kind=4). and stores a new value back to the first argument location.Device code atomicmax( mem. mem. These functions are: Table 3. Counting Atomic Functions Function atomicinc( mem. imax ) Compare and Swap Atomic Function This atomic function reads and returns the value of the first argument. mem. The function is: 28 . value ) atomicand( mem.and.value) mem = ior(mem.9. and atomically stores a new value back to the first argument location if the first and second argument are equal.value) mem = ieor(mem. It also compares the first argument with the second argument.

SIZEOF Intrinsic A call to sizeof(A). j j = sizeof(i) ! this assigns the value 4 to j 29 .val) Additional Atomic Update if (mem == comp) then mem = val endif Restrictions This section lists restrictions on statements and features that can appear in device subprograms. • Objects with character type must have LEN=1. • Input/Output statements are not allowed: READ. Host code Host subprograms may use intrinsic functions. INQUIRE. Compare and Swap Atomic Function Function atomiccas(mem. • ENTRY statements are not allowed. Reference Table 3. • Fortran intrinsic functions not listed in Section 3. integer(kind=4) :: i. BACKSPACE. • Floating point exception handling is not supported. CLOSE.comp.Chapter 3. • Objects with the Pointer and Allocatable attribute are not allowed. NAMELIST. • Automatic arrays must be fixed size. WRITE. • STOP and PAUSE statements are not allowed. • Optional arguments are not allowed. • Subroutine and function calls are supported only if they can be inlined. character substrings are not supported.10. ENDFILE.3 are not supported. where A is a variable or expression. such as the new sizeof intrinsic function. • Cray pointers are not supported. PRINT. REWIND. OPEN. • Recursive subroutines and functions are not allowed.6. • Alternate return specifications are not allowed. FORMAT. returns the number of bytes required to hold the value of A.

30 .

” on page 41. intent(out) :: numdev cudaGetDeviceCount assigns the number of available devices to its first argument.Chapter 4. the runtime initializes and connects to the device the first time a runtime routine is called or a device array is allocated. Most of the runtime API routines are integer functions that return an error code. Runtime APIs The system module cudafor defines the interfaces to the Runtime API routines. cudaSetDeviceFlags integer function cudaSetDevice( flags ) integer. refer to “Error Handling. Device Management Use the functions in this section for device management. they return a value of zero if the call was successful. To interpret the error codes. be aware that initialization can add some overhead. intent(in) :: flags cudaSetDeviceFlags records how the CUDA runtime interacts with this host thread. Tip When doing timing runs. Initialization No explicit initialization is required. and a nonzero value if there was an error. 31 . cudaSetDevice integer function cudaSetDevice( devnum ) integer. cudaGetDeviceCount integer function cudaGetDeviceCount( numdev ) integer. intent(in) :: devnum cudaSetDevice selects the device to associate with this host thread.

thus creating potential read-after-write. Any subsequent CUDA calls or operations will reinitialize the runtime. Consistent use of TYPE(C_PTR) and 32 . or write-after-write hazards for some of these memory accesses. as arguments to simplify interfacing to existing CUDA C code. cudaGetDeviceProperties integer function cudaGetDeviceProperties( prop. defined in the cudafor module. It may return an error condition if one of the preceding operations fails.Thread Management cudaGetDevice integer function cudaGetDevice( devnum ) integer. it is implicitly called when the host thread exits. Calling cudaThreadExit is optional. intent(out) :: devnum type(cudadeviceprop). provided through the Fortran 2003 iso_c_binding module. Memory Management Many of the memory management routines can take device arrays as arguments. intent(in) :: prop cudaChooseDevice assigns the device number that best matches the properties given in prop to its first argument. as TYPE(C_DEVPTR). intent(out) :: prop integer. Some can also take C types. use the functions in this section for thread management. To avoid these potential issues. intent(in) :: devnum cudaGetDeviceProperties returns the properties of a given device. cudaThreadExit integer function cudaThreadExit() cudaThreadExit explicitly cleans up all runtime-related CUDA resources associated with the host thread. CUDA Fortran has extended the F2003 derived type TYPE(C_PTR) by providing a C device pointer. Thread Management Sometimes threads within a block access the same addresses in shared or global memory. devnum ) type(cudadeviceprop). cudaThreadSynchronize integer function cudaThreadSynchronize() cudaThreadSynchronize blocks execution of the host subprogram until all preceding kernels and operations are complete. prop ) integer. write-after-read. intent(out) :: devnum cudaGetDevice assigns the device number associated with this host thread to its first argument. cudaChooseDevice integer function cudaChooseDevice ( devnum.

1. should be of benefit. Runtime APIs TYPE(C_DEVPTR). Use the functions in this section for memory management. “Device Code Intrinsic Datatypes. The height is an integer. Both of these features are subject to change when. pitch. it is possible to construct a Fortran device array out of a TYPE(C_DEVPTR) by using an extension of the iso_c_binding subroutine c_f_pointer. as well as consistency checks between Fortran device arrays and host arrays. two-dimensional device array of a supported type specified in Table 3.” on page 21.1. devptr may be any allocatable. “Device Code Intrinsic Datatypes. and in effect transfer the allocation to the Fortran array. cdesc. devptr may be any allocatable. devptr may be any allocatable device array of a supported type specified in Table 3. Currently. devptr may be of TYPE(C_DEVPTR). The width is in terms of number of elements. Under CUDA Fortran.” on page 21. cudaFree integer function cudaFree(devptr) cudaFree deallocates data on the device. a shape as the third argument. count) cudaMalloc allocates data on the device. cudaFreeArray integer function cudaFreeArray(carray) type(cudaArrayPtr) :: carray 33 . c_f_pointer will take a TYPE(C_DEVPTR) as the first argument.1. width. Or. width. in the future.” on page 21. cudaMalloc integer function cudaMalloc(devptr. an allocatable device array as the second argument. in which case the integer values are expressed in bytes.Chapter 4. cudaMallocArray integer function cudaMallocArray(carray. height) cudaMallocPitch allocates data on the device. one-dimensional device array of a supported type specified in Table 3. in which case the count is in bytes. cudaMallocPitch integer function cudaMallocPitch(devptr. devptr may also be of TYPE(C_DEVPTR). Or. Similarly. and the padded width is returned in the variable pitch. “Device Code Intrinsic Datatypes. proper Fortran pointers for device data are supported. The count is in terms of elements. there is also a function C_DEVLOC() defined which will create a TYPE(C_DEVPTR) that holds the C address of the Fortran device array argument. height) type(cudaArrayPtr) :: carray type(cudaChannelFormatDesc) :: cdesc integer :: width. height cudaMallocArray allocates a data array on the device. cudaMallocPitch may pad the data. devptr may be of TYPE(C_DEVPTR).

kdir may be optional. cudaMemset2D integer function cudaMemset2D(devptr. scalar or array. cudaMemcpyAsync integer function cudaMemcpyAsync(dst. width. spitch. value. pitch. in which case the pitch. The count is in terms of elements. width. kdir may be optional.1. cudaMemcpyDeviceToHost.1. dst and src may be any device or host array. The width 34 . dpitch. height.1. “Device Code Intrinsic Datatypes. The copy can be associated with a stream by passing a non-zero stream argument.” on page 21. dst and src may be any device or host.” on page 21.1. Alternatively. dst and src may be of TYPE(C_DEVPTR) or TYPE(C_PTR). cudaMemcpy integer function cudaMemcpy(dst. refer to “Data Transfer Using Runtime Routines. count. “Device Code Intrinsic Datatypes. height) cudaMemset2D sets an array to the specified value. If kdir is specified. The value must match in type and kind. cudaMemcpyDeviceToHost. it must be one of the defined enums cudaMemcpyHostToDevice.1. kdir) cudaMemcpy copies data from one location to another. “Device Code Intrinsic Datatypes. src. Or. or cudaMemcpyDeviceToDevice.” on page 20. devptr may be of TYPE(C_DEVPTR). src. otherwise the stream argument is optional and defaults to zero.” on page 21. devptr may be any device array of a supported type specified in Table 3. stream) cudaMemcpy copies data from one location to another. devptr may be of TYPE(C_DEVPTR). in which case the count is in term of bytes. The pitch. count) cudaMemset sets a location or array to the specified value. of a supported type specified in Table 3. devptr may be any device scalar or array of a supported type specified in Table 3. dst and src may be any device or host. The count is in terms of elements. width. for more information. scalar or array. “Device Code Intrinsic Datatypes. “Device Code Intrinsic Datatypes. of a supported type specified in Table 3. The value must match in type and kind. or cudaMemcpyDeviceToDevice. Or. cudaMemset integer function cudaMemset(devptr. refer to “Data Transfer Using Runtime Routines. in which case the count is in term of bytes. dst and src may be of TYPE(C_DEVPTR) or TYPE(C_PTR). kdir.” on page 21. and the lowest byte of value is used.” on page 20. for more information. value. and height are in terms of elements. and height are in terms of bytes. of a supported type specified in Table 3. src.” on page 21. in which case the count is in term of bytes. width. This function operates on page-locked host memory only. The count is in terms of elements. it must be one of the defined enums cudaMemcpyHostToDevice. Alternatively. and the lowest byte of value is used. kdir) cudaMemcpy2D copies data from one location to another. count.Memory Management cudaFreeArray frees an array that was allocated on the device. If kdir is specified. cudaMemcpy2D integer function cudaMemcpy2D(dst.

otherwise the stream argument is optional and defaults to zero. cudaMemcpy2DAsync integer function cudaMemcpy2DAsync(dst. dst and src may be of TYPE(C_DEVPTR) or TYPE(C_PTR). srca. it must be one of the defined enums cudaMemcpyHostToDevice. srcy. srcx. dsty. width. dsty. srcx. Runtime APIs and height are in terms of elements. srcx. in which case the width and height are in term of bytes. kdir may be optional. kdir may be optional. kdir) type(cudaArrayPtr) :: srca integer :: dpitch. width. width. dpitch. Alternatively. If kdir is specified. kdir cudaMemcpy2DToArray copies array data to and from the device. src. height. cudaMemcpyDeviceToHost.Chapter 4. dsty. height. width. kdir) type(cudaArrayPtr) :: dsta integer :: dstx. height. in which case the width and height are in term of bytes. dstx. dpitch. If kdir is specified. 35 . srca. height.1. it must be one of the defined enums cudaMemcpyHostToDevice. dst and src may be of TYPE(C_DEVPTR) or TYPE(C_PTR). dsty. kdir) type(cudaArrayPtr) :: dsta integer :: dstx. for more information. cudaMemcpy2DFromArray integer function cudaMemcpy2DFromArray(dst. kdir cudaMemcpy2DFromArray copies array data to and from the device. kdir) type(cudaArrayPtr) :: srca integer :: dstx. cudaMemcpy2DToArray integer function cudaMemcpy2DToArray(dsta. src. count. kdir cudaMemcpyToArray copies array data to and from the device. height. count. This function operates on page-locked host memory only. stream) cudaMemcpy2D copies data from one location to another. “Device Code Intrinsic Datatypes. cudaMemcpyDeviceToHost. count. Alternatively. dstx. count. dst and src may be any device or host array. or cudaMemcpyDeviceToDevice.” on page 20. spitch. spitch. of a supported type specified in Table 3. width. src.” on page 21. The copy can be associated with a stream by passing a non-zero stream argument. or cudaMemcpyDeviceToDevice. kdir cudaMemcpyFromArray copies array data to and from the device. srcy. kdir. refer to “Data Transfer Using Runtime Routines. The width and height are in terms of elements.” on page 20. refer to “Data Transfer Using Runtime Routines. for more information. srcy. spitch. cudaMemcpyFromArray integer function cudaMemcpyFromArray(dst. dsty. cudaMemcpyToArray integer function cudaMemcpyToArray(dsta.

which was allocated with cudaMalloc3D to a specified value. width.” on page 21.Memory Management cudaMemcpyArrayToArray integer function cudaMemcpyArrayToArray(dsta. count. kdir) type(cudaArrayPtr) :: dsta. Alternatively. srca. srcy. dsty. cext is also a derived type which holds the extents of the allocated array. height. cext) type(cudaPitchedPtr). srcx. srcx. the extents in each dimension specified by cext. cudaMalloc3DArray integer function cudaMalloc3DArray(carray. srcy. kdir) type(cudaArrayPtr) :: dsta. intent(out) :: pitchptr type(cudaExtent). pitchptr may be any allocatable. pitchptr is a derived type defined in the cudafor module. three-dimensional device array of a supported type specified in “Datatypes allowed. dstx. intent(in) :: cext cudaMalloc3D allocates data on the device. cudaMemcpy2DArrayToArray integer function cudaMemcpy2DArrayToArray(dsta. dsty. dstx. cext) type(cudaArrayPtr) :: carray type(cudaChannelFormatDesc) :: cdesc type(cudaExtent) :: cext cudaMalloc3DArray allocates array data on the device. kdir cudaMemcpy2DArrayToArray copies array data to and from the device. cudaMemcpy3D integer function cudaMemcpy3D(p) type(cudaMemcpy3DParms) :: p cudaMemcpy3D copies elements from one 3D array to another as specified by the data held in the derived type p. 36 . srcx. srca integer :: dstx. cext) type(cudaPitchedPtr) :: pitchptr integer :: value type(cudaExtent) :: cext cudaMemset3D sets elements of an array. height. srca integer :: dstx. count. cdesc. srca. srcy. value. cudaMalloc3D integer function cudaMalloc3D(pitchptr. dsty. width. dsty. cudaMemset3D integer function cudaMemset3D(pitchptr. srcx. srcy. kdir cudaMemcpyArrayToArray copies array data to and from the device.

symbol. src. src may be any host scalar or array of a supported type as specified in “Datatypes allowed. kdir cudaMemcpyToSymbol copies data from the source to a device area in global or constant memory space referenced by a symbol. count. cudaMemcpyToSymbol integer function cudaMemcpyToSymbol(symbol. This function operates on page-locked host memory only.” on page 21. The copy can be associated with a stream by passing a non-zero stream argument.” on page 21. cudaMemcpyFromSymbolAsync integer function cudaMemcpyFromSymbolAsync(dst. kdir integer. symbol. count. kdir. src. kdir integer. dst may be any host scalar or array of a supported type specified in “Datatypes allowed. count. offset. offset. src may be any host scalar or array of a supported type specified in “Datatypes allowed. offset. cudaMemcpyFromSymbol integer function cudaMemcpyFromSymbol(dst. stream) type(cudaSymbol) :: symbol integer :: count. kdir. kdir cudaMemcpyFromSymbol copies data from a device area in global or constant memory space referenced by a symbol to a destination on the host. This function operates on page-locked host memory only. stream) type(cudaMemcpy3DParms) :: p integer :: stream cudaMemcpy3DAsync copies elements from one 3D array to another as specified by the data held in the derived type p.” on page 21. stream) type(cudaSymbol) :: symbol integer :: count. offset. offset. kdir) type(cudaSymbol) :: symbol integer :: count. count. cudaMemcpyToSymbolAsync integer function cudaMemcpyToSymbolAsync(symbol. offset.Chapter 4. offset. The copy can be associated with a stream by passing a non-zero stream argument. Runtime APIs cudaMemcpy3DAsync integer function cudaMemcpy3D(p. optional :: stream cudaMemcpyToSymbol copies data from the source to a device area in global or constant memory space referenced by a symbol. The count is in terms of elements. The count is in terms of elements. kdir) type(cudaSymbol) :: symbol integer :: count. The count is in terms of elements. optional :: stream 37 . offset.

flags) type(C_DEVPTR) :: devptr type(C_PTR) :: hostptr integer :: flags cudaHostGetDevicePointer returns a pointer to a device memory address corresponding to the pinned memory on the host. device :: vx(:) csvx = “vx” Istat = cudaGetSymbolAddress(cdvx. cudaHostAlloc integer function cudaHostAlloc(hostptr. symbol) integer :: size type(cudaSymbol) :: symbol cudaGetSymbolSize sets the variable size to the size of a device area in global or constant memory space referenced by the symbol. The normal iso_c_binding subroutine c_f_pointer can be used to move the type(c_ptr) to a Fortran pointer. Size is in bytes. 100) Vx = 0.” on page 21. flags cudaHostAlloc allocates pinned memory on the host. vx. A symbol can be set to an external device name via a character string. hostptr. symbol) type(C_DEVPTR) :: devptr type(cudaSymbol) :: symbol cudaGetSymbolAddress returns in the devptr argument the address of symbol on the device. dst may be any host scalar or array of a supported type specified in “Datatypes allowed. allocatable. The flags argument enables different options to be specified that affect the allocation. flags) type(C_PTR) :: hostptr integer :: size. hostptr is a pinned memory buffer that was allocated via cudaHostAlloc(). or returns an error if the memory is unavailable. and read and written by. The copy can be associated with a stream by passing a non-zero stream argument. size. cudaGetSymbolAddress integer function cudaGetSymbolAddress(devptr. The following code sequence initializes a global device array “vx” from a CUDA C kernel: type(cudaSymbol) :: csvx type(c_devptr) :: cdvx real. csvx) Call c_f_pointer(cdvx. cudaHostGetDevicePointer integer function cudaHostGetDevicePointer(devptr. The count is in terms of elements.Memory Management cudaMemcpyFromSymbol copies data from a device area in global or constant memory space referenced by a symbol to a destination on the host. It returns in hostptr the address of the pagelocked allocation. This function operates on page-locked host memory only. a kernel which runs on 38 . It returns in devptr an address that can be passed to.0 cudaGetSymbolSize integer function cudaGetSymbolSize(size.

intent(in) :: stream cudaStreamDestroy releases any resources associated with the given stream. size is in bytes. asynchronous operations. The normal iso_c_binding subroutine c_f_pointer can be used to move the type(c_ptr) to a Fortran pointer. It may also return another error condition if some asynchronous operations failed. The normal iso_c_binding subroutine c_f_pointer can be used to move the type(c_devptr)to a device array. cudaStreamSynchronize integer function cudaStreamSynchronize( stream ) integer. 39 . Runtime APIs the device. It returns in hostptr the address of the page-locked allocation. intent(in) :: stream cudaStreamQuery tests whether all operations enqueued to the selected stream are complete. size) type(C_PTR) :: hostptr integer :: size cudaMallocHost allocates pinned memory on the host. cudaStreamQuery integer function cudaStreamQuery( stream ) integer. cudaMallocHost integer function cudaMallocHost(hostptr.Chapter 4. The flags argument is provided for future releases. intent(out) :: stream cudaStreamCreate creates an asynchronous stream and assigns its identifier to its first argument. and the value cudaErrorNotReady if not. cudaFreeHost integer function cudaFreeHost(hostptr) type(C_PTR) :: hostptr cudaFreeHost deallocates pinned memory on the host allocated with cudaMalloHost. intent(in) :: stream cudaStreamSynchronize blocks execution of the host subprogram until all preceding kernels and operations associated with the given stream are complete. cudaStreamDestroy integer function cudaStreamDestroy( stream ) integer. Stream Management Use the functions in this section for stream management. It may return error codes from previous. it returns zero (success) if all operations are complete. or returns an error if the memory is unavailable. cudaStreamCreate integer function cudaStreamCreate( stream ) integer.

intent(out) :: event cudaEventCreate creates an event object and assigns the event identifier to its first argument cudaEventRecord integer function cudaEventRecord( event. cudaEventCreate integer function cudaEventCreate( event ) type(cudaEvent). end cudaEventElapsedTime computes the elapsed time between two events (in milliseconds). It returns cudaErrorInvalidValue if cudaEventRecord has not been called for this event. intent(in) :: event cudaEventDestroy destroys the resources associated with an event object. cudaEventElapsedTime integer function cudaEventElapsedTime( time. end) float :: time type(cudaEvent). It returns success (zero) if the event has been recorded. cudaEventDestroy integer function cudaEventDestroy( event ) type(cudaEvent). If stream is zero. intent(in) :: stream cudaEventRecord issues an operation to the given stream to record an event. This function is only valid with events recorded on stream zero. cudaEventQuery integer function cudaEventQuery( event ) type(cudaEvent). intent() :: start. stream ) type(cudaEvent). 40 .Event Management Event Management Use the functions in this section to manage events. cudaEventSynchronize integer function cudaEventSynchronize( event ) type(cudaEvent). intent(in) :: event cudaEventSynchronize blocks until the event has been recorded. and cudaErrorNotReady if it has not. start. intent(in) :: event cudaEventQuery tests whether an event has been recorded. intent(in) :: event integer. It returns cudaErrorInvalidValue if either event has not yet been recorded. the event is recorded after all preceding operations in all streams are complete. It returns a value of cudaErrorInvalidValue if cudaEventRecord has not been called for this event. The event is recorded after all preceding operations in the stream are complete.

cudaGetLastError integer function cudaGetLastError() cudaGetLastError returns the error code that was most recently returned from any runtime call in this host thread. Runtime APIs Error Handling Use the functions in this section for error handling. cudaDriverGetVersion integer function cudaDriverGetVersion(iversion) integer :: iversion cudaDriverGetVersion returns the version number of the installed CUDA driver as iversion. This function automatically returns cudaErrorInvalidValue if the iversion argument is NULL. then it returns 0 as iversion. 41 . cudaGetErrorString function cudaGetErrorString( errcode ) integer. intent(in) :: errcode character*(*) :: cudaGetErrorString cudaGetErrorString returns the message string associated with the given error code. Version Management Use the functions in this section for version management. If no driver is installed.Chapter 4. cudaRuntimeGetVersion integer function cudaRuntimeGetVersion(iversion) integer :: iversion cudaRuntimeGetVersion returns the version number of the installed CUDA Runtime as iversion. This function automatically returns cudaErrorInvalidValue if the iversion argument is NULL.

42 .

For simplicity.M). C is then NxL attributes(global) subroutine mmul_kernel( A. then moves to the next submatrix of A rowwise and of B columnwise. ty ! submatrices stored in shared memory real. tx. B(M. Source Code Listing ! start the module containing the matmul kernel module mmul_mod use cudafor contains ! mmul_kernel computes A*B into C where ! A is NxM. • Each thread within the block computes one element of the submatrix.16) ! the value of C(i.pgroup. it reads a block submatrix of A and a block submatrix of B. accumulates the submatrix product. C(N.Chapter 5. L ) real :: A(N. N. The program caches the submatrices of A and B in the fast shared memory.htm Matrix Multiplication Example This example shows a program to compute the product C of two matrices A and B. M. The submatrix size is chosen so the number of threads in a block is a multiple of the warp size (32) and is less than the maximum number of threads per thread block (512). Examples This chapter contains For up-to-date information about the state of the current release.j) being computed 43 . the program assumes the matrix sizes are a multiple of 16. visit the frequently asked questions (FAQ) section of the pgroup. C. Each element of the result is the product of one row of A by one column of B. value :: N. shared :: Asub(16. and has not been highly optimized for execution time. B is MxL.16). kb. Bsub(16.com/support/index. B.com web page at: www. L integer :: i. M. j. as follows: • Each thread block computes one 16x16 submatrix of C.L).L) integer. k. The program computes the products by accumulating submatrix products.

2 ) ! Allocate the device arrays allocate( Adev(N. Cdev ) end subroutine mmul end module mmul_mod N 44 .j) ! Wait until all elements are filled call syncthreads() ! Multiply the two submatrices ! Each of the 16x16 threads accumulates the ! dot product for its element of C(i. Bdev.L) ) ! Copy A and B to the device Adev = A(1:N. C ! allocatable device arrays real. 1 ) call mmul_kernel<<<dimGrid.k) * Bsub(k. dimension(:. B.dimBlock>>>( Adev.ty) = B(kb+tx-1. 2 ) L = size( B.j)) i = (blockidx%x-1) * 16 + tx j = (blockidx%y-1) * 16 + ty Cij = 0. device. Cdev(N. dimBlock ! Get the array sizes N = size( A. B. Bdev. C ) real.j) = sum(A(i. Bdev(M. L/16. allocatable. dimension(:.kb+ty-1) Bsub(tx.ty) = A(i.16 Cij = Cij + Asub(tx.1:L) = Cdev deallocate( Adev.:) * B(:.:) :: A. the block size do kb = 1. ! Copy the results back and free up memory C(1:N. 16. 16 ! Fill the submatrices ! Each of the 16x16 threads in the thread block ! loads one element of Asub and Bsub Asub(tx.0 ! Do the k loop in chunks of 16.ty) enddo ! Synchronize to make sure all threads are done ! reading the submatrices before overwriting them ! in the next iteration of the kb loop call syncthreads() enddo ! Each of the 16x16 threads stores its element ! to the global C array C(i. 1 ) M = size( A.M).Bdev. M.Cdev ! dim3 variables to define the grid and block shapes type(dim3) :: dimGrid.j) do k = 1. Cdev.Matrix Multiplication Example real :: Cij ! Get the thread indices tx = threadidx%x ty = threadidx%y ! This thread computes C(i.:) :: Adev.:) = B(1:M. 1 ) dimBlock = dim3( 16.1:L) ! Create the grid and block dimensions dimGrid = dim3( N/16.j) = Cij end subroutine mmul_kernel ! The host routine to drive the matrix multiplication subroutine mmul( A.L).1:M) Bdev(:.

Examples Source Code Description This source code module mmul_mod has two subroutines. MMUL_KERNEL This kernel subroutine has two device memory input arrays. and three scalars giving the array sizes. • It frees the device memory arrays. • It accumulates the dot product of its row and column of the submatrices. Bdev. • It determines the i and j indices. • It allocates device memory arrays Adev. no extra synchronization is needed between the copy operations and the kernel launch. it does the following steps: • It loads one element of the submatrices of A and B into shared memory. 45 . passed as assumed-shape arrays. and one output array. C. • It calls mmul_kernel to compute Cdev on the device. • It synchronizes again to make sure all threads are done reading the submatrices before starting the next block. This routine computes the dot product of A(i. The thread executing this routine is one of 16x16 threads cooperating in a thread block. • It initializes a scalar in which it will accumulate the dot product. • It copies the arrays A and B to Adev and Bdev using array assignments. for which element of C(i. • For each block.Chapter 5. it stores the computed value into the correct element of C. and L. and Cdev. A and B. • It fills dimGrid and dimBlock to hold the grid and thread block sizes. C. • It copies Cdev back from device memory to C. MMUL This host subroutine has two input arrays. • Finally. one device memory output array. It performs the following operations: • It determines the thread indices for this thread. • It synchronizes to make sure both submatrices are loaded by all threads in the block.j) it is computing.j) for a particular value of i and j. The host subroutine mmul is a wrapper for the kernel routine mmul_kernel. The routine performs the following operations: • It determines the size of the matrices in N. Because the data copy operations are synchronous. A and B. • It steps through the arrays A and B in blocks of size 16. depending on the block and thread index. M.:)*B(:.

46 .

pgroup. 47 .htm.pgroup.pgroup. PGI documentation is available at www. Two Centerpointe Drive Lake Oswego.com/userforum/index.com/resources/docs.pgroup.com trs@pgroup.com www. The forum newsgroups may contain answers to commonly asked questions.php Or contact us electronically using any of the following means: Fax Sales Support WWW +1-503-682-2637 sales@pgroup. Inc.Chapter 6.com/support/faq.pgroup. Log in to the PGI website to access the forum: www. Phone support is not currently available.com All technical support is by email or submissions using an online form at www.com/support.htm.htm or in your local copy of the documentation in the release directory doc/index. Many questions and problems can be resolved at our frequently asked questions (FAQ) site at www. OR 97035 USA The PGI User Forum is monitored by members of the PGI engineering and support teams as well as other PGI customers. Contact Information You can contact The Portland Group at: The Portland Group STMicroelectronics.

48 .

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->