You are on page 1of 566

Programming under ChorusOS

Jean-Marie Riet
University Paris VII - Denis Diderot
November 14, 2000
Contents
1 General presentation 3
1.1 The development of Chorus . . . . . . . . . . . . . . . . . . . . 3
1.1.1 History of the project . . . . . . . . . . . . . . . . . . . . 3
1.1.2 The Sun Embedded Workshop . . . . . . . . . . . . . . . 4
1.2 Con gurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 The cross development environment . . . . . . . . . . . . . . . . 5
1.4 Main characteristics of ChorusOS . . . . . . . . . . . . . . . . . 5
1.5 The main abstractions of ChorusOS . . . . . . . . . . . . . . . . 6
1.6 The architecture of ChorusOS . . . . . . . . . . . . . . . . . . . 8
1.6.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.6.2 The nucleus layer . . . . . . . . . . . . . . . . . . . . . . 9
1.6.2.1 The micro-kernel . . . . . . . . . . . . . . . . . 9
1.6.2.2 The drivers . . . . . . . . . . . . . . . . . . . . 10
1.6.2.3 The PD actor . . . . . . . . . . . . . . . . . . . 10
1.6.3 The OS layer . . . . . . . . . . . . . . . . . . . . . . . . 10
1.6.3.1 Overview . . . . . . . . . . . . . . . . . . . . . 10
1.6.3.2 The C INIT actor . . . . . . . . . . . . . . . . 11
1.6.3.3 The IOM actor . . . . . . . . . . . . . . . . . . 11
1.6.3.4 The AM actor . . . . . . . . . . . . . . . . . . 11
1.6.3.5 The ADMIN actor . . . . . . . . . . . . . . . . 11
1.6.3.6 The SHM REP actor . . . . . . . . . . . . . . . 12
1.6.3.7 The RDBC actor . . . . . . . . . . . . . . . . . 12
1.6.4 The application layer . . . . . . . . . . . . . . . . . . . . 12
1.6.5 The di erent features . . . . . . . . . . . . . . . . . . . . 12
1.6.5.1 Actor management features . . . . . . . . . . . 12
1.6.5.2 Scheduling option . . . . . . . . . . . . . . . . . 13
1.6.5.3 The memory management features . . . . . . . 13
1.6.5.4 The time management features . . . . . . . . . 13
1.6.5.5 The inter-thread synchronization features . . . 14
3
4 Contents
1.6.5.6 The local invocation optional features . . . . . 14
1.6.5.7 The communication optional features . . . . . . 14
1.6.5.8 The le system optional features . . . . . . . . 15
1.6.5.9 The debugging features . . . . . . . . . . . . . 15
1.6.5.10 The C INIT optional feature . . . . . . . . . . 16
1.6.5.11 The I/O management options . . . . . . . . . . 16
1.6.5.12 The network features . . . . . . . . . . . . . . . 16
1.6.5.13 The administration features . . . . . . . . . . . 16
1.6.6 The restart option . . . . . . . . . . . . . . . . . . . . . 17
2 Building and booting a ChorusOS system 19
2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2 Installing the delivery . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2.1 Downloading the les . . . . . . . . . . . . . . . . . . . . 19
2.2.2 Installing the packages . . . . . . . . . . . . . . . . . . . 20
2.2.3 Organization of the installed les tree . . . . . . . . . . . 20
2.2.3.1 The rst level . . . . . . . . . . . . . . . . . . . 20
2.2.3.2 The tools directory . . . . . . . . . . . . . . . 21
2.2.3.3 The kernel directory . . . . . . . . . . . . . . 22
2.2.3.4 The os directory . . . . . . . . . . . . . . . . . 23
2.2.3.5 The src directory . . . . . . . . . . . . . . . . 23
2.3 Building a standard image . . . . . . . . . . . . . . . . . . . . . 24
2.3.1 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.2 The configure utility . . . . . . . . . . . . . . . . . . . 24
2.3.3 Building the system image . . . . . . . . . . . . . . . . . 25
2.3.4 Building the root le system . . . . . . . . . . . . . . . . 26
2.4 Con guring and tuning a system image . . . . . . . . . . . . . . 28
2.4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.4.2 The con guration les . . . . . . . . . . . . . . . . . . . 28
2.4.3 Con guration tools . . . . . . . . . . . . . . . . . . . . . 30
2.4.3.1 The graphical tool ews . . . . . . . . . . . . . . 30
2.4.3.2 Command-line tool: configurator . . . . . . . 30
2.5 Booting the ChorusOS system . . . . . . . . . . . . . . . . . . . 34
2.5.1 Preparing the boot monitor . . . . . . . . . . . . . . . . 35
2.5.2 Setting up booting services on the host station . . . . . . 36
2.5.3 The RARP service . . . . . . . . . . . . . . . . . . . . . 36
2.5.4 The TFTP service . . . . . . . . . . . . . . . . . . . . . 37
2.5.5 Connecting the console and booting . . . . . . . . . . . . 38
Contents 5
3 Getting started 41
3.1 The C INIT actor and its initialization . . . . . . . . . . . . . . 41
3.1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.1.2 The sysadm.ini initialization le . . . . . . . . . . . . . 41
3.1.3 Mounting the root le system . . . . . . . . . . . . . . . 42
3.1.4 About security: the /etc/security le . . . . . . . . . . 43
3.1.5 The startup les . . . . . . . . . . . . . . . . . . . . . . 44
3.2 List of C INIT built-in commands . . . . . . . . . . . . . . . . . 44
3.3 The echo command . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.4 The environment: env, setenv and unsetenv . . . . . . . . . . 46
3.5 Statistics: memstat and chorusStat . . . . . . . . . . . . . . . 46
3.5.1 The memstat command . . . . . . . . . . . . . . . . . . . 46
3.5.2 The chorusStat command . . . . . . . . . . . . . . . . . 46
3.6 The mount and umount commands . . . . . . . . . . . . . . . . . 47
3.7 The aps command . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.8 The arun command . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.8.1 General form of the arun command . . . . . . . . . . . . 49
3.8.2 What the command does . . . . . . . . . . . . . . . . . . 49
3.8.3 Executing standard commands: the example of cs . . . . 50
3.8.3.1 The cs command without argument . . . . . . 50
3.8.3.2 Getting information on system con guration . . 51
3.8.3.3 Getting the system's statistics . . . . . . . . . . 52
3.8.3.4 Getting unique identi ers . . . . . . . . . . . . 52
3.8.3.5 Getting information on scheduling . . . . . . . 53
3.8.3.6 Getting information about one thread . . . . . 53
3.8.3.7 Getting information about IPC ports . . . . . . 53
3.8.3.8 Getting information about one actor . . . . . . 54
3.8.4 Dynamic loading of user applications . . . . . . . . . . . 54
3.9 The akill command . . . . . . . . . . . . . . . . . . . . . . . . 59
3.10 The ping command . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.11 The reboot command . . . . . . . . . . . . . . . . . . . . . . . 59
3.12 The kernel debugger: kdb . . . . . . . . . . . . . . . . . . . . . 60
3.12.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.12.1.1 Entering the kernel debugger . . . . . . . . . . 60
3.12.1.2 Exiting the kernel debugger . . . . . . . . . . . 60
3.12.1.3 The help command . . . . . . . . . . . . . . . . 60
3.12.2 List of kdb commands . . . . . . . . . . . . . . . . . . . 60
6 Contents
4 Producing ChorusOS applications 63
4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.2 Organization of the working space . . . . . . . . . . . . . . . . . 64
4.3 General conventions for writing C or C++ codes for ChorusOS . 64
4.3.1 Header les . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.3.2 Symbolic constants . . . . . . . . . . . . . . . . . . . . . 66
4.3.3 Prede ned types . . . . . . . . . . . . . . . . . . . . . . 66
4.3.4 Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.3.5 Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.4 General principles . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.4.1 The libraries . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.4.2 Entry point . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.4.3 Supervisor binaries . . . . . . . . . . . . . . . . . . . . . 68
4.4.4 User binaries . . . . . . . . . . . . . . . . . . . . . . . . 68
4.5 The production's environment . . . . . . . . . . . . . . . . . . . 68
4.5.1 The make environment . . . . . . . . . . . . . . . . . . . 68
4.5.2 The imake environment . . . . . . . . . . . . . . . . . . 69
4.5.3 Producing binaries . . . . . . . . . . . . . . . . . . . . . 71
4.5.3.1 The ChorusOSMkMf command . . . . . . . . . . 71
4.5.3.2 Producing the dependencies . . . . . . . . . . . 71
4.5.3.3 Producing the binary . . . . . . . . . . . . . . . 72
4.5.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.5.4.1 Producing examples . . . . . . . . . . . . . . . 72
4.5.4.2 Building a library and using it . . . . . . . . . 74
4.6 Dynamic actors . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.6.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.6.2 Building a dynamic library . . . . . . . . . . . . . . . . . 78
4.6.2.1 The DynamicLibraryTarget macro . . . . . . . 78
4.6.2.2 Example . . . . . . . . . . . . . . . . . . . . . . 79
4.6.3 Using a dynamic library . . . . . . . . . . . . . . . . . . 79
4.6.3.1 Dynamic link at actor startup . . . . . . . . . . 79
4.6.3.2 Dynamic programming . . . . . . . . . . . . . . 80
4.6.4 Running a dynamic program . . . . . . . . . . . . . . . . 82
5 General services 83
5.1 Naming in ChorusOS . . . . . . . . . . . . . . . . . . . . . . . . 83
5.1.1 Unique identi ers . . . . . . . . . . . . . . . . . . . . . . 83
5.1.1.1 De nition and characteristics . . . . . . . . . . 83
5.1.1.2 The KnUniqueId type . . . . . . . . . . . . . . 84
Contents 7
5.1.1.3 Clearing a unique identi er . . . . . . . . . . . 84
5.1.1.4 Comparing unique identi ers . . . . . . . . . . 84
5.1.1.5 Sites and unique identi ers . . . . . . . . . . . 85
5.1.1.6 Building a unique identi er . . . . . . . . . . . 85
5.1.2 Capabilities . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.1.2.1 De nition and characteristics . . . . . . . . . . 86
5.1.2.2 The KnCap type . . . . . . . . . . . . . . . . . . 86
5.1.3 Local identi ers . . . . . . . . . . . . . . . . . . . . . . . 87
5.1.4 Getting the local site number . . . . . . . . . . . . . . . 87
5.1.5 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
5.2 Getting information about ChorusOS . . . . . . . . . . . . . . . 88
5.2.1 Getting a tunable parameter . . . . . . . . . . . . . . . . 88
5.2.2 Accessing the environment . . . . . . . . . . . . . . . . . 90
5.2.2.1 Getting a value: sysGetEnv . . . . . . . . . . . 90
5.2.2.2 Setting a value: sysSetEnv . . . . . . . . . . . 91
5.2.2.3 Deleting a value: sysUnsetEnv . . . . . . . . . 91
5.2.2.4 Example . . . . . . . . . . . . . . . . . . . . . . 91
5.3 Input/output from the system console . . . . . . . . . . . . . . 93
5.3.1 Reading from the system console: sysRead . . . . . . . . 93
5.3.2 Writing to the system console: sysWrite . . . . . . . . . 93
5.3.3 Polling the system console: sysPoll . . . . . . . . . . . 94
5.3.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.4 Rebooting the local system . . . . . . . . . . . . . . . . . . . . . 95
6 Actors 97
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6.2 Actors attributes . . . . . . . . . . . . . . . . . . . . . . . . . . 99
6.2.1 Identifying an actor . . . . . . . . . . . . . . . . . . . . . 99
6.2.1.1 Capability of an actor . . . . . . . . . . . . . . 99
6.2.1.2 Symbolic name . . . . . . . . . . . . . . . . . . 99
6.2.1.3 The case of c actors . . . . . . . . . . . . . . . 100
6.2.2 Actors states . . . . . . . . . . . . . . . . . . . . . . . . 100
6.2.2.1 The basic states . . . . . . . . . . . . . . . . . 100
6.2.2.2 Changing the basic state of an actor . . . . . . 101
6.2.3 Actors privilege . . . . . . . . . . . . . . . . . . . . . . . 101
6.2.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
6.3 Getting status of actors on the site . . . . . . . . . . . . . . . . 105
6.3.1 Getting status of all actors on the site . . . . . . . . . . 105
6.3.2 Getting status of all c actors on the site . . . . . . . . . 106
8 Contents
6.4 Creating actors . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
6.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 108
6.4.2 Promoting an actor to the status of c actor . . . . . . . . 109
6.4.3 High level creation of c actors . . . . . . . . . . . . . . . 109
6.4.3.1 Speci c attributes of a c actor . . . . . . . . . . 109
6.4.3.2 Spawning a c actor: the afexec primitives . . . 110
6.4.4 Low level operations on c actors . . . . . . . . . . . . . . 118
6.4.4.1 Creating an empty c actor: acreate . . . . . . 118
6.4.4.2 Loading text and data for a c actor: aload . . 120
6.4.4.3 Activating a c actor: astart . . . . . . . . . . 125
6.4.5 The low level interface: actorCreate . . . . . . . . . . . 126
6.5 Deleting actors . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
6.5.1 The low level interface: actorDelete . . . . . . . . . . . 129
6.5.2 Terminating c actors . . . . . . . . . . . . . . . . . . . . 130
6.5.2.1 Termination of the main thread . . . . . . . . . 130
6.5.2.2 Killing a c actor: akill . . . . . . . . . . . . . 131
6.6 Synchronizing c actors' threads' executions . . . . . . . . . . . . 133
7 Basic memory management 139
7.1 General concepts . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.1.1 The memory models and the features . . . . . . . . . . . 139
7.1.2 Memory management . . . . . . . . . . . . . . . . . . . . 141
7.1.2.1 Supervisor and user address spaces . . . . . . . 141
7.1.2.2 The page size . . . . . . . . . . . . . . . . . . . 141
7.1.2.3 Parameters for memory management . . . . . . 142
7.2 Organization of an actor's address space . . . . . . . . . . . . . 144
7.2.1 The concept of region . . . . . . . . . . . . . . . . . . . . 144
7.2.2 Organization of user c actors' address space . . . . . . . 144
7.2.2.1 The MEM FLAT model . . . . . . . . . . . . . 144
7.2.2.2 The MEM PROTECTED model . . . . . . . . 146
7.2.3 Organization of supervisor c actors . . . . . . . . . . . . 147
7.2.4 Regions' descriptors . . . . . . . . . . . . . . . . . . . . . 148
7.2.5 Regions' attributes . . . . . . . . . . . . . . . . . . . . . 149
7.2.5.1 Initialization attributes . . . . . . . . . . . . . 149
7.2.5.2 Protection attributes . . . . . . . . . . . . . . . 149
7.2.5.3 Virtual memory speci c attributes . . . . . . . 151
7.2.6 Virtual addresses and physical addresses . . . . . . . . . 151
7.2.7 Displaying all regions of an actor . . . . . . . . . . . . . 154
7.2.7.1 The rgnStat function . . . . . . . . . . . . . . 154
Contents 9
7.2.7.2 Examples . . . . . . . . . . . . . . . . . . . . . 155
7.3 The Posix interface . . . . . . . . . . . . . . . . . . . . . . . . . 157
7.4 Creating a new region . . . . . . . . . . . . . . . . . . . . . . . 158
7.4.1 The rgnAllocate primitive . . . . . . . . . . . . . . . . 158
7.4.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 159
7.4.2.1 Internal allocation . . . . . . . . . . . . . . . . 159
7.4.2.2 External allocation . . . . . . . . . . . . . . . . 162
7.5 Freeing regions in an actor . . . . . . . . . . . . . . . . . . . . . 166
7.5.1 Implicit deallocation at actor termination . . . . . . . . . 166
7.5.2 The rgnFree primitive . . . . . . . . . . . . . . . . . . . 167
7.5.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
7.6 Advanced operations on regions . . . . . . . . . . . . . . . . . . 170
7.6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 170
7.6.2 Sharing memory . . . . . . . . . . . . . . . . . . . . . . 171
7.6.2.1 The rgnMapFromActor primitive . . . . . . . . 171
7.6.2.2 Example . . . . . . . . . . . . . . . . . . . . . . 172
7.6.3 Inheriting a copy of existing regions . . . . . . . . . . . . 178
7.6.3.1 The rgnInitFromActor primitive . . . . . . . . 178
7.6.3.2 Example . . . . . . . . . . . . . . . . . . . . . . 179
7.7 Manipulating supervisor address space . . . . . . . . . . . . . . 181
7.7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 181
7.7.2 Displaying all regions in supervisor address space . . . . 182
7.7.3 The supervisor address space memory allocator . . . . . 183
7.7.4 Safely accessing supervisor address space . . . . . . . . . 187
7.7.4.1 The svMemRead primitive . . . . . . . . . . . . 187
7.7.4.2 The svMemWrite primitive . . . . . . . . . . . . 187
7.7.4.3 Example . . . . . . . . . . . . . . . . . . . . . . 188
7.8 The POSIX shared memory . . . . . . . . . . . . . . . . . . . . 191
7.8.1 The POSIX SHM feature and the SHM REP actor . . . . . . 191
7.8.2 Creating/opening a shared memory object . . . . . . . . 192
7.8.3 Unlinking a shared memory object: shm unlink . . . . . 192
7.8.4 Setting the size of a shared object: ftruncate . . . . . . 193
7.8.5 Mapping a shared memory object: mmap . . . . . . . . . 193
7.8.6 Unmapping a shared memory object: munmap . . . . . . 194
8 Threads 195
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
8.2 Threads names and states . . . . . . . . . . . . . . . . . . . . . 198
8.2.1 Thread identi cation . . . . . . . . . . . . . . . . . . . . 198
10 Contents
8.2.1.1 Local identi cation . . . . . . . . . . . . . . . . 198
8.2.1.2 Symbolic name . . . . . . . . . . . . . . . . . . 198
8.2.2 Thread states . . . . . . . . . . . . . . . . . . . . . . . . 199
8.3 Threads scheduling . . . . . . . . . . . . . . . . . . . . . . . . . 200
8.3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 200
8.3.2 The scheduling classes . . . . . . . . . . . . . . . . . . . 200
8.3.2.1 The FIFO class . . . . . . . . . . . . . . . . . . 201
8.3.2.2 The RR class . . . . . . . . . . . . . . . . . . . 201
8.3.2.3 The RT class . . . . . . . . . . . . . . . . . . . 202
8.3.3 Getting or setting scheduling information . . . . . . . . . 203
8.3.3.1 The default KnThreadDefaultSched class . . . 203
8.3.3.2 The threadScheduler primitive . . . . . . . . 203
8.3.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 204
8.3.4.1 Getting scheduling attributes . . . . . . . . . . 204
8.3.4.2 Modifying scheduling attributes . . . . . . . . . 206
8.3.4.3 Fifo scheduling vs Round Robin scheduling . . 207
8.4 Delaying a thread . . . . . . . . . . . . . . . . . . . . . . . . . . 208
8.4.1 The threadDelay primitive . . . . . . . . . . . . . . . . 208
8.4.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
8.5 Creating and deleting a thread . . . . . . . . . . . . . . . . . . . 211
8.5.1 Creating a thread . . . . . . . . . . . . . . . . . . . . . . 211
8.5.1.1 The threadCreate primitive . . . . . . . . . . 211
8.5.1.2 Examples . . . . . . . . . . . . . . . . . . . . . 213
8.5.2 Deleting a thread . . . . . . . . . . . . . . . . . . . . . . 225
8.5.2.1 The e ect of calling exit and exit . . . . . . 225
8.5.2.2 The threadDelete primitive . . . . . . . . . . 226
8.5.2.3 Examples . . . . . . . . . . . . . . . . . . . . . 226
8.6 Activating a thread . . . . . . . . . . . . . . . . . . . . . . . . . 229
8.7 Stopping and restarting a thread . . . . . . . . . . . . . . . . . 230
8.7.1 Stopping a thread . . . . . . . . . . . . . . . . . . . . . . 230
8.7.2 Restarting a stopped thread . . . . . . . . . . . . . . . . 230
8.8 Aborting a thread . . . . . . . . . . . . . . . . . . . . . . . . . . 230
8.8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 230
8.8.2 The complete states diagram of a thread . . . . . . . . . 231
8.8.3 The threadAbort and threadAborted primitives . . . . 232
8.8.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 232
8.8.4.1 A command for aborting a given thread . . . . 232
8.8.4.2 Thread's auto-abortion . . . . . . . . . . . . . . 233
8.8.4.3 A non abortable call . . . . . . . . . . . . . . . 233
Contents 11
8.8.4.4 Testing the aborted state . . . . . . . . . . . . 234
8.9 Getting the attributes of all threads in an actor . . . . . . . . . 235
8.9.1 The threadStat primitive . . . . . . . . . . . . . . . . . 235
8.9.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
8.10 Execution times . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
8.11 Managing a thread's context . . . . . . . . . . . . . . . . . . . . 239
8.12 A complete example: a fork function . . . . . . . . . . . . . . . 240
8.12.1 Objectives and general principles . . . . . . . . . . . . . 240
8.12.2 Variations on solutions . . . . . . . . . . . . . . . . . . . 241
8.12.2.1 A solution with creation of a c actor . . . . . . 241
8.12.2.2 A solution with low level actor's creation . . . . 245
8.13 Posix threads (pthreads) . . . . . . . . . . . . . . . . . . . . . . 246
8.13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 246
8.13.2 Posix threads' attributes . . . . . . . . . . . . . . . . . . 246
8.13.3 Creating a pthread . . . . . . . . . . . . . . . . . . . . . 247
8.13.4 Yielding the processor . . . . . . . . . . . . . . . . . . . 248
8.13.5 Terminating a pthread . . . . . . . . . . . . . . . . . . . 248
8.13.6 Killing a pthread . . . . . . . . . . . . . . . . . . . . . . 248
8.13.7 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
9 Synchronization 251
9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
9.2 Thread semaphores . . . . . . . . . . . . . . . . . . . . . . . . . 252
9.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 252
9.2.2 Initializing a thread semaphore . . . . . . . . . . . . . . 252
9.2.3 Posting a thread semaphore . . . . . . . . . . . . . . . . 253
9.2.4 Waiting on a thread semaphore . . . . . . . . . . . . . . 253
9.2.5 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
9.3 General mutexes . . . . . . . . . . . . . . . . . . . . . . . . . . 256
9.3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 256
9.3.2 Initializing a mutex . . . . . . . . . . . . . . . . . . . . . 256
9.3.3 Acquiring a mutex . . . . . . . . . . . . . . . . . . . . . 257
9.3.4 Freeing a mutex . . . . . . . . . . . . . . . . . . . . . . . 257
9.3.5 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
9.4 Real-time mutexes . . . . . . . . . . . . . . . . . . . . . . . . . 259
9.5 General semaphores . . . . . . . . . . . . . . . . . . . . . . . . . 261
9.5.1 Initializing a semaphore: semInit . . . . . . . . . . . . . 261
9.5.2 Waiting for a semaphore: semP . . . . . . . . . . . . . . 262
9.5.3 Signaling a semaphore: semV . . . . . . . . . . . . . . . . 262
12 Contents
9.5.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
9.6 Event sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
9.6.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 267
9.6.2 Initializing an event set: eventInit . . . . . . . . . . . . 267
9.6.3 Posting an event: eventPost . . . . . . . . . . . . . . . 268
9.6.4 Waiting for events: eventWait . . . . . . . . . . . . . . . 268
9.6.5 Clearing events: eventClear . . . . . . . . . . . . . . . . 269
9.6.6 Example: a synchronization barrier's implementation . . 270
9.7 Posix synchronization mechanism . . . . . . . . . . . . . . . . . 276
9.7.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 276
9.7.2 Waiting for thread termination . . . . . . . . . . . . . . 277
9.7.3 Posix mutexes . . . . . . . . . . . . . . . . . . . . . . . . 277
9.7.3.1 Attributes of Posix mutexes . . . . . . . . . . . 277
9.7.3.2 Locking a Posix mutex . . . . . . . . . . . . . . 278
9.7.3.3 Unlocking a Posix mutex . . . . . . . . . . . . . 278
9.7.4 Posix condition variables . . . . . . . . . . . . . . . . . . 278
9.7.4.1 Initializing a condition variable . . . . . . . . . 278
9.7.4.2 Waiting on a condition variable . . . . . . . . . 279
9.7.4.3 Signaling a condition variable . . . . . . . . . . 279
9.7.4.4 General outline . . . . . . . . . . . . . . . . . . 280
10 Private data 281
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
10.2 Standard services . . . . . . . . . . . . . . . . . . . . . . . . . . 282
10.2.1 The errno variable . . . . . . . . . . . . . . . . . . . . . 282
10.2.2 Threads' local identi ers . . . . . . . . . . . . . . . . . . 284
10.3 The private data keys . . . . . . . . . . . . . . . . . . . . . . . . 284
10.4 Chorus per-thread private data . . . . . . . . . . . . . . . . . . 285
10.4.1 Creating and deleting a thread speci c data key . . . . . 285
10.4.2 Setting a thread-speci c value . . . . . . . . . . . . . . . 286
10.4.3 Getting a thread speci c value . . . . . . . . . . . . . . . 287
10.4.4 Deleting all thread speci c values . . . . . . . . . . . . . 287
10.4.5 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
10.5 Chorus per-actor private data . . . . . . . . . . . . . . . . . . . 291
10.5.1 Creating and deleting an actor private key . . . . . . . . 291
10.5.2 Setting an actor speci c value . . . . . . . . . . . . . . . 292
10.5.3 Getting an actor speci c value . . . . . . . . . . . . . . . 292
10.5.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
10.6 Posix private data . . . . . . . . . . . . . . . . . . . . . . . . . . 296
Contents 13
10.6.1 Creating a Posix thread-speci c key . . . . . . . . . . . . 296
10.6.2 Deleting a Posix thread-speci c key . . . . . . . . . . . . 297
10.6.3 Associating a value to a Posix thread-speci c key . . . . 297
10.6.4 Getting the value for a Posix thread-speci c key . . . . . 297
11 Local access points (LAPs) 299
11.1 General overview . . . . . . . . . . . . . . . . . . . . . . . . . . 299
11.2 General tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
11.2.1 Copying data between user and supervisor address spaces 301
11.2.2 Manipulating lap descriptors . . . . . . . . . . . . . . . . 302
11.2.2.1 Initializing a lap descriptor to zero . . . . . . . 302
11.2.2.2 Testing if a lap descriptor is initialized . . . . . 302
11.2.2.3 Duplicating a lap descriptor . . . . . . . . . . . 303
11.3 The server's point of view . . . . . . . . . . . . . . . . . . . . . 303
11.3.1 Creating a local access point . . . . . . . . . . . . . . . . 303
11.3.2 Binding a symbolic name to a lap descriptor . . . . . . . 304
11.3.3 Unbinding a lap's symbolic name . . . . . . . . . . . . . 304
11.3.4 Getting handler invoker . . . . . . . . . . . . . . . . . . 305
11.3.5 Deleting a local access point: svLapDelete . . . . . . . . 305
11.3.6 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 305
11.3.6.1 Installing a lap in the current actor . . . . . . . 305
11.3.6.2 Copy service between user and supervisor spaces 306
11.3.6.3 De ning di erent lap handlers in the same actor 309
11.4 The client's point of view . . . . . . . . . . . . . . . . . . . . . . 312
11.4.1 Resolving a name . . . . . . . . . . . . . . . . . . . . . . 312
11.4.2 Synchronous invocation of a lap handler . . . . . . . . . 312
11.4.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 313
11.4.3.1 A client of the lapCp service . . . . . . . . . . . 313
11.4.3.2 A client of the multiLap service . . . . . . . . . 315
12 Time management 321
12.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
12.1.1 The features . . . . . . . . . . . . . . . . . . . . . . . . . 321
12.1.2 Describing time intervals . . . . . . . . . . . . . . . . . . 321
12.1.3 Time resolution and system time . . . . . . . . . . . . . 322
12.2 The DATE feature . . . . . . . . . . . . . . . . . . . . . . . . . . 323
12.2.1 Getting universal time resolution: univTimeGetRes . . . 323
12.2.2 Getting time of day: univTime . . . . . . . . . . . . . . 323
12.2.3 Setting time of day: univTimeSet, univTimeAdjust . . . 323
14 Contents
12.2.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 324
12.3 Chorus timers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
12.3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 325
12.3.2 Initializing a thread pool object . . . . . . . . . . . . . . 326
12.3.3 Creating a Chorus timer . . . . . . . . . . . . . . . . . . 326
12.3.4 Deleting a Chorus timer . . . . . . . . . . . . . . . . . . 327
12.3.5 Starting, canceling or querying a timer: timerSet . . . . 327
12.3.6 Blocking a thread on a thread pool object . . . . . . . . 328
12.3.7 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
12.4 The VTIMER feature . . . . . . . . . . . . . . . . . . . . . . . . . 333
12.4.1 Virtual timeouts . . . . . . . . . . . . . . . . . . . . . . 333
12.4.2 Setting a virtual timeout . . . . . . . . . . . . . . . . . . 334
12.4.2.1 The general interface . . . . . . . . . . . . . . . 334
12.4.2.2 Setting an actor's virtual timeout . . . . . . . . 335
12.4.2.3 Setting a thread's virtual timeout . . . . . . . . 336
12.4.3 Canceling a virtual timeout . . . . . . . . . . . . . . . . 336
12.4.3.1 The general interface . . . . . . . . . . . . . . . 336
12.4.3.2 Cancelling an actor's virtual timeout . . . . . . 337
12.4.3.3 Canceling a thread's virtual timeout . . . . . . 337
12.4.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 337
12.4.4.1 The compute function . . . . . . . . . . . . . . 337
12.4.4.2 A general service . . . . . . . . . . . . . . . . . 340
12.4.4.3 The server's point of view . . . . . . . . . . . . 340
12.4.4.4 The client's point of view . . . . . . . . . . . . 345
12.4.4.5 Execution . . . . . . . . . . . . . . . . . . . . . 346
12.5 Posix interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
12.5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 350
12.5.2 General services . . . . . . . . . . . . . . . . . . . . . . . 350
12.5.3 Posix timers . . . . . . . . . . . . . . . . . . . . . . . . . 351
12.5.3.1 Identifying a Posix timer . . . . . . . . . . . . . 351
12.5.3.2 Creating a Posix timer . . . . . . . . . . . . . . 351
12.5.3.3 Deleting a Posix timer . . . . . . . . . . . . . . 352
12.5.3.4 Arming a Posix timer . . . . . . . . . . . . . . 352
13 Communications 355
13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
13.2 Mailboxes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
13.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 356
13.2.2 Creating a message space . . . . . . . . . . . . . . . . . . 356
Contents 15
13.2.3 Opening a message space . . . . . . . . . . . . . . . . . . 358
13.2.4 Operations on messages . . . . . . . . . . . . . . . . . . 358
13.2.4.1 General principles . . . . . . . . . . . . . . . . 358
13.2.4.2 Allocating a message . . . . . . . . . . . . . . . 359
13.2.4.3 Posting a message to a queue . . . . . . . . . . 360
13.2.4.4 Getting a message from a queue . . . . . . . . . 361
13.2.4.5 Freeing a message . . . . . . . . . . . . . . . . 361
13.2.4.6 Removing a message from a queue . . . . . . . 362
13.2.5 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
13.3 The ChorusOS IPC . . . . . . . . . . . . . . . . . . . . . . . . . 368
13.3.1 The features . . . . . . . . . . . . . . . . . . . . . . . . . 368
13.3.1.1 The IPC feature . . . . . . . . . . . . . . . . . . 368
13.3.1.2 The IPC REMOTE feature . . . . . . . . . . . . . 369
13.3.2 Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
13.3.2.1 Overview . . . . . . . . . . . . . . . . . . . . . 371
13.3.2.2 Identi cation . . . . . . . . . . . . . . . . . . . 372
13.3.2.3 Port states . . . . . . . . . . . . . . . . . . . . 373
13.3.2.4 Creating a port . . . . . . . . . . . . . . . . . . 374
13.3.2.5 Deleting a port . . . . . . . . . . . . . . . . . . 375
13.3.3 Ports groups . . . . . . . . . . . . . . . . . . . . . . . . . 376
13.3.3.1 Introduction . . . . . . . . . . . . . . . . . . . 376
13.3.3.2 Allocating a port group capability . . . . . . . 377
13.3.3.3 Inserting a port into a port group . . . . . . . . 378
13.3.3.4 Removing a port from a port group . . . . . . . 379
13.3.4 Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . 379
13.3.4.1 Message structure . . . . . . . . . . . . . . . . 379
13.3.4.2 Current message . . . . . . . . . . . . . . . . . 380
13.3.5 Destinations and modes . . . . . . . . . . . . . . . . . . 381
13.3.6 Asynchronous communication . . . . . . . . . . . . . . . 382
13.3.6.1 Overview . . . . . . . . . . . . . . . . . . . . . 382
13.3.6.2 Sending a message . . . . . . . . . . . . . . . . 382
13.3.6.3 Receiving a message . . . . . . . . . . . . . . . 384
13.3.6.4 Replying to a message . . . . . . . . . . . . . . 386
13.3.6.5 Example . . . . . . . . . . . . . . . . . . . . . . 386
13.3.7 Synchronous communications . . . . . . . . . . . . . . . 393
13.3.7.1 Overview . . . . . . . . . . . . . . . . . . . . . 393
13.3.7.2 Initiating an RPC . . . . . . . . . . . . . . . . 393
13.3.7.3 Sending a reply . . . . . . . . . . . . . . . . . . 395
13.3.7.4 Example . . . . . . . . . . . . . . . . . . . . . . 395
16 Contents
13.3.8 Manipulating the current message . . . . . . . . . . . . . 398
13.3.8.1 Getting information about the current message 398
13.3.8.2 Getting the current message body . . . . . . . . 399
13.3.8.3 Saving the current message . . . . . . . . . . . 399
13.3.8.4 Restoring a saved message as current message . 400
13.4 Message handlers . . . . . . . . . . . . . . . . . . . . . . . . . . 401
13.4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 401
13.4.2 Connecting and disconnecting a handler to a port . . . . 402
13.4.3 Exiting a message handler . . . . . . . . . . . . . . . . . 403
13.4.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 403
13.5 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
13.5.1 Getting a port sequence number . . . . . . . . . . . . . . 408
13.5.2 Migrating a port . . . . . . . . . . . . . . . . . . . . . . 408
13.5.2.1 The service . . . . . . . . . . . . . . . . . . . . 408
13.5.2.2 Example . . . . . . . . . . . . . . . . . . . . . . 409
14 Abortions, exceptions and traps 413
14.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
14.2 Abort handlers . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
14.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 414
14.2.2 Consulting the abort handlers vector . . . . . . . . . . . 417
14.2.3 Connecting abort handlers . . . . . . . . . . . . . . . . . 417
14.2.4 Disconnecting an abort handler . . . . . . . . . . . . . . 418
14.2.5 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 418
14.3 Exception handlers . . . . . . . . . . . . . . . . . . . . . . . . . 421
14.3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 421
14.3.2 Consulting the exception handlers vector . . . . . . . . . 422
14.3.3 Connecting an exception handler . . . . . . . . . . . . . 423
14.3.4 Disconnecting an exception handler . . . . . . . . . . . . 424
14.3.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 424
14.3.5.1 Example 1 . . . . . . . . . . . . . . . . . . . . . 424
14.3.5.2 Example 2: a server for automatic allocation . . 430
14.4 Traps and subsystems . . . . . . . . . . . . . . . . . . . . . . . 438
14.4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 438
14.4.2 Trap handlers . . . . . . . . . . . . . . . . . . . . . . . . 439
14.4.2.1 General presentation . . . . . . . . . . . . . . . 439
14.4.2.2 Connecting a trap handler . . . . . . . . . . . . 440
14.4.2.3 Disconnecting a trap handler . . . . . . . . . . 440
14.4.2.4 Getting a trap handler . . . . . . . . . . . . . . 441
Contents 17
14.4.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
14.5 Timeout handlers . . . . . . . . . . . . . . . . . . . . . . . . . . 446
14.5.1 Setting a timeout . . . . . . . . . . . . . . . . . . . . . . 446
14.5.2 Canceling a timeout . . . . . . . . . . . . . . . . . . . . 447
14.5.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 447
15 The MEM VIRTUAL memory management model 449
15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449
15.1.1 General overview . . . . . . . . . . . . . . . . . . . . . . 449
15.1.2 Segments . . . . . . . . . . . . . . . . . . . . . . . . . . 450
15.1.3 Local caches . . . . . . . . . . . . . . . . . . . . . . . . . 451
15.1.4 The generic interface . . . . . . . . . . . . . . . . . . . . 451
15.2 Basic concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451
15.2.1 General aspects . . . . . . . . . . . . . . . . . . . . . . . 451
15.2.2 Parameters for memory management . . . . . . . . . . . 452
15.2.3 Regions' attributes . . . . . . . . . . . . . . . . . . . . . 453
15.2.3.1 Paging attributes . . . . . . . . . . . . . . . . . 453
15.2.3.2 Inheritance attributes . . . . . . . . . . . . . . 454
15.2.3.3 Opaque attributes . . . . . . . . . . . . . . . . 454
15.3 Organization of c actors . . . . . . . . . . . . . . . . . . . . . . 455
15.4 Duplicating an actor address space . . . . . . . . . . . . . . . . 456
15.4.1 The rgnDup primitive . . . . . . . . . . . . . . . . . . . . 456
15.4.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . 457
15.4.2.1 The application . . . . . . . . . . . . . . . . . . 457
15.4.2.2 The case of user actors . . . . . . . . . . . . . . 458
15.4.2.3 The case of supervisor actors . . . . . . . . . . 461
15.5 Regions and segments . . . . . . . . . . . . . . . . . . . . . . . 461
15.5.1 The KnObjDesc type . . . . . . . . . . . . . . . . . . . . 461
15.5.2 Creating and initializing a region from a segment . . . . 462
15.5.3 Creating a region and mapping a segment on it . . . . . 463
15.6 Explicit access to a segment . . . . . . . . . . . . . . . . . . . . 464
15.6.1 Reading data from a segment . . . . . . . . . . . . . . . 464
15.6.2 Writing data to a segment . . . . . . . . . . . . . . . . . 464
15.7 Explicit access to a local cache . . . . . . . . . . . . . . . . . . . 465
15.7.1 Finding or creating a local cache . . . . . . . . . . . . . 465
15.7.2 Getting statistics of a cache . . . . . . . . . . . . . . . . 466
15.7.3 Flushing and controlling access to a local cache . . . . . 466
15.7.4 Initializing a local cache . . . . . . . . . . . . . . . . . . 468
15.7.5 Reading data through a local cache . . . . . . . . . . . . 468
18 Contents
15.7.6 Writing data through a local cache . . . . . . . . . . . . 469
15.8 The generic interface . . . . . . . . . . . . . . . . . . . . . . . . 469
15.8.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 469
15.8.2 The MpCreate transaction . . . . . . . . . . . . . . . . 470
15.8.2.1 A request . . . . . . . . . . . . . . . . . . . . . 470
15.8.2.2 A reply . . . . . . . . . . . . . . . . . . . . . . 470
15.8.3 MpRelease transaction . . . . . . . . . . . . . . . . . . 470
15.8.3.1 A request . . . . . . . . . . . . . . . . . . . . . 471
15.8.3.2 A reply . . . . . . . . . . . . . . . . . . . . . . 471
15.8.4 MpGetAccess transaction . . . . . . . . . . . . . . . . 471
15.8.4.1 A request . . . . . . . . . . . . . . . . . . . . . 471
15.8.4.2 A reply . . . . . . . . . . . . . . . . . . . . . . 472
15.8.5 MpPullIn transaction . . . . . . . . . . . . . . . . . . . 472
15.8.5.1 A request . . . . . . . . . . . . . . . . . . . . . 472
15.8.5.2 A reply . . . . . . . . . . . . . . . . . . . . . . 473
15.8.6 MpPushOut transaction . . . . . . . . . . . . . . . . . 473
15.8.6.1 A request . . . . . . . . . . . . . . . . . . . . . 473
15.8.6.2 A reply . . . . . . . . . . . . . . . . . . . . . . 473
15.9 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474
15.9.1 Dealing with kernel requests . . . . . . . . . . . . . . . . 474
15.9.2 A mapper . . . . . . . . . . . . . . . . . . . . . . . . . . 478
15.9.3 A client using rgnMap . . . . . . . . . . . . . . . . . . . . 480
15.9.4 Clients using sgRead or sgWrite . . . . . . . . . . . . . 485
15.9.5 Client using sgFlush . . . . . . . . . . . . . . . . . . . . 487
16 The ChorusOS driver framework 489
16.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489
16.1.1 General overview . . . . . . . . . . . . . . . . . . . . . . 489
16.1.2 Hardware organization . . . . . . . . . . . . . . . . . . . 491
16.1.3 Software organization . . . . . . . . . . . . . . . . . . . . 491
16.2 The components of the framework . . . . . . . . . . . . . . . . . 493
16.2.1 General organization . . . . . . . . . . . . . . . . . . . . 493
16.2.2 The device tree . . . . . . . . . . . . . . . . . . . . . . . 493
16.2.3 The driver registry module . . . . . . . . . . . . . . . . . 494
16.2.4 The device registry module . . . . . . . . . . . . . . . . 496
16.2.5 Files organization . . . . . . . . . . . . . . . . . . . . . . 498
16.2.6 General tools . . . . . . . . . . . . . . . . . . . . . . . . 500
16.2.6.1 Message logging . . . . . . . . . . . . . . . . . 500
16.2.6.2 An Imakefile . . . . . . . . . . . . . . . . . . 500
Contents 19
16.3 DKI common services . . . . . . . . . . . . . . . . . . . . . . . . 500
16.3.1 The synchronization service . . . . . . . . . . . . . . . . 500
16.3.2 Memory allocation . . . . . . . . . . . . . . . . . . . . . 501
16.3.2.1 Allocating/freeing space . . . . . . . . . . . . . 501
16.3.3 Allocating physical memory . . . . . . . . . . . . . . . . 501
16.3.4 The device tree . . . . . . . . . . . . . . . . . . . . . . . 501
16.3.4.1 Browsing the device tree . . . . . . . . . . . . . 501
16.3.4.2 Modifying the device tree . . . . . . . . . . . . 502
16.3.4.3 Nodes properties . . . . . . . . . . . . . . . . . 503
16.3.4.4 Examples . . . . . . . . . . . . . . . . . . . . . 506
16.3.4.5 Device tree high-level services . . . . . . . . . . 510
16.3.5 Driver registry . . . . . . . . . . . . . . . . . . . . . . . 511
16.3.5.1 Registering a driver . . . . . . . . . . . . . . . 511
16.3.5.2 Accessing the driver registry . . . . . . . . . . . 511
16.3.5.3 Example . . . . . . . . . . . . . . . . . . . . . . 512
16.3.6 Device registry . . . . . . . . . . . . . . . . . . . . . . . 513
16.3.6.1 Services dedicated to device drivers . . . . . . . 513
16.3.6.2 Services for driver clients . . . . . . . . . . . . 514
16.3.7 Other services . . . . . . . . . . . . . . . . . . . . . . . . 516
16.3.7.1 Timeouts . . . . . . . . . . . . . . . . . . . . . 516
16.3.7.2 Busy wait . . . . . . . . . . . . . . . . . . . . . 517
16.3.7.3 System event management . . . . . . . . . . . . 517
16.3.7.4 Global interrupt masking . . . . . . . . . . . . 517
16.3.7.5 Thread preemption disabling . . . . . . . . . . 517
16.4 DKI speci c services . . . . . . . . . . . . . . . . . . . . . . . . 518
16.4.1 Speci c input/output services . . . . . . . . . . . . . . . 518
16.4.2 Processor family speci c services . . . . . . . . . . . . . 518
16.5 Device Driver Interface (DDI) . . . . . . . . . . . . . . . . . . . 518
16.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 518
16.5.2 Common bus services . . . . . . . . . . . . . . . . . . . . 519
16.5.3 Common bus properties . . . . . . . . . . . . . . . . . . 519
16.5.4 Common bus driver events . . . . . . . . . . . . . . . . . 519
16.5.5 Handlers . . . . . . . . . . . . . . . . . . . . . . . . . . . 520
16.5.6 Common bus driver service routines . . . . . . . . . . . . 521
A System calls error codes 525
B Index and list of prede ned constants 527
20 Contents
C Index and list of prede ned types 531
D Index and list of primitives 533
Preface
About this book
This book is a programmer's guide: it presents the main aspects of the system
you will face when starting using it. It intends to help you to develop your own
applications for ChorusOS.
The text is divided into
Part 1 (chapters 1-2) describes the overall structure of the system and how a
system can be built, con gured and booted;
Part 2 (chapters 3-4) shows how to create applications that can be executed
under ChorusOS.
Part 3 (chapters 5-13) presents the abstarctions of the system and the di er-
ents APIs which are provided to develop applications intended to be executed
under ChorusOS (actors and threads, memory management, synchronization,
communications and cross actor invocation).
Part 4 (chapters 14-16) presents advanced concepts (handlers, virtual memory
and drivers).

To help you along, this book provides numerous examples written in C. I have
tried to keep these examples as short and as simple as possible so that they are
readable.
A compressed archive (tar/gzip) of the various include les and source les of
this book is located at
http://www.pps.jussieu.fr/~rifflet/CHORUS/examples.gz
You can send me messages electronically at
rifflet@pps.jussieu.fr
Finally, for more information about ChorusOS see
http://www.sun.com/software/chorusos

1
2 Preface
Chapter 1
General presentation
1.1 The development of Chorus
1.1.1 History of the project
The Chorus system started life in the 80's as a research project at INRIA (Insti-
tut National de Recherche en Informatique et Automatisme) which is a French
public computing research institute. Chorus aimed at distribution, integrating
communication mechanisms at the heart of the system. Chorus V2 brought a
rst distributed Unix V7 layer within the system.
The "Chorus systemes" company was founded at the beginning of 1987 by Hu-
bert Zimmermann, Michel Gien and a small team. This led to the Chorus/MiX
V3.2 system being released in 1989; this was a distributed, real-time system pro-
viding binary compatibility with SCO systems on Intel based machines. Various
avors of this system have been used within di erent PABX's of Alcatel. Other
avors of the system have been used to equip JVC karaoke machines in Japan.
In 1989, Chorus systemes started a partnership with Unisys in order to provide
a Single System Image Unix system on top of MPP machines. In the same
time frame, the Unix basis evolved from SVR3.2 to SVR4.0. The Unisys OPUS
machine ran this type of system, the ICL Goldrush machine also ran a variant
of that system.
A joint-development e ort set up with Novell led to the development of a Chorus
based SVR4.2 ES/MP system which was abandoned when Novell ceased all Unix
development activities.
In 1994, Chorus systemes focussed its activities on the micro-kernel at the heart
of the system, targeting real-time embedded applications. Since then, a small
3
4 Chapter 1. General presentation
OS layer providing some limited POSIX API's based on FreeBSD software has
been used to deliver the ChorusOS system. This has been used in some large
environments, mainly by Telecom Equipment Manufacturers such as Lucent,
Nortel and others. However, it is still used in some low-end devices such as
webphones (with a Java environment), set-top boxes or point of sales devices.
In 1997, Chorus systems, and its 80 employees in France, USA and Japan, has
been acquired by Sun Microsystems Inc.
1.1.2 The Sun Embedded Workshop
The Sun Embedded Workshop (SEW) provides as a rst component a set of
tools and utilities for developing and managing applications intended to run
on a high-performance operating system (the Chorus Operating System) which
constitutes the second component of the workshop. The rst component is a
host-target development environment and the second one (ChorusOS) is a real-
time modular operating system. This book is based on SEW 2.1 and the 4.0
release of ChorusOS.
1.2 Con gurations
A typical installation of the SEW requires a minimum hardware con guration
consisting of two machines connected by a Ethernet network or a serial line:
 a target PC/AT which will run a ChorusOS operating system instance and
applications. The hardware prerequisites for this PC are as follows:
- a processor of one of the following types: Intel 386, 486 or Pentium;
- an Ethernet board (SMC EtherEZ or Novell NE2000) for downloading the
system and applications which otherwise will be loaded from a diskette;
- approximately 4 megabytes of main memory;
- possibly a oppy disk (3 1/4 inch) to boot the system.
Of course SEW runs on hardware con gurations other than this trivial but
common example. Boards or machines based on SPARC or PPC may also be
used;
 a host development machine which will be used for building the ChorusOS
system bootable image, for developing applications and downloading these ap-
plications. The following host machines may be used:
- Sun/Sparc station with Solaris system (version >2.5) or SunOS (version
>4.1);
- PC/AT with Microsoft Windows NT 4.0.
1.3. The cross development environment 5
This type of basic con guration corresponds to the following diagram:
le system

User applications
NFS Applications
System and
applications rsh tftp rarp operating system

CHORUS kernel
target machine host machine

network

The con guration may be more complicated: for example the boot server used
for loading the system and the host used for development may be separate
machines. Furthermore, other machines may be used to NFS mount speci c le
systems.

1.3 The cross development environment


The SEW provides complete support for developing C and C++ applications
intended to run on a ChorusOS machine. It includes:
 a C and C++ development tool-chain comprising GNU gcc and g++ cross
compilers (generating executable code on the host system for a given type of
target machine);
 a C and C++ symbolic debugger: XRAY for ChorusOS;
 an embedded debugger for system applications;
 con guration tools for building system archives (described in chapter 2);
 di erent libraries: thread safe C++, thread safe ANSI-C, a subset of POSIX
1003.1a, POSIX 1003.1b (timers, message queues, semaphores and shared mem-
ory), POSIX 1003.1c (Posix threads), thread safe mathematical ANSI-C, C++
iostream, BSD sockets, X11;
 application management tools: netboot (for remotely booting the Chorus-
OS system on the target machine), remote shell, default console, monitoring,
logging, pro ling and benchmarking.

1.4 Main characteristics of ChorusOS


It is modular: it can either scale down to very small implementations for em-
bedding or scale up to POSIX platforms.
6 Chapter 1. General presentation
It is highly con gurable: the system is divided into modules which cooperate
in order to o er di erent features, e.g. speci c functionalities of the system.
When building a system, it is possible to select the features which will be inte-
grated into the system. Thus, the system may be adjusted to meet the exact
needs of the applications which are intended to run on it. This way, memory
may be saved and performance improved. Furthermore, each module de nes
system parameters which a ect the behavior of the system: these parameters
may be set to speci c values when building the system.
It is a real-time system: scheduling policies are based on xed priorities of
the components of an application, kernel calls are bounded in time, interrupt
latency is kept minimal and it is possible to attach interrupt handlers to an
application.
It supports some level of transparent distribution: it allows applications
running on di erent machines to cooperate in a transparent way inside a set of
machines interconnected on a network and constituting a Chorus domain.
It incorporates several features which allow high availability of the system: these
features include memory protection mechanisms, hot restart allowing applica-
tions to resume execution after a system crash from a consistent state, and
dynamic recon guration.

1.5 The main abstractions of ChorusOS


We rst outline the naming schemes used in the system (they will be presented
in more detail in chapter 5):
 unique identi ers: in the current version, they consist of 64 bit objects (a
32 bit head and a 32 bit tail);
 capabilities: they are composed of a unique identi er and a key which is
also a 64 bit object (a 32 bit head and a 32 bit tail);
 local identi ers: these small integers may be interpreted in the context of
a given application.
Let us now brie y summarize the main abstractions provided by the system,
which we will study in more detail in later chapters of this book.
 actors: from the system's point of view, an actor is the unit of resource en-
capsulation. An actor is a repository of resources such as an address space or
ports for communication. In other words, it de nes a virtual machine. From
the user's point of view, an actor constitutes the unit of modularization of ap-
plications. Actors are identi ed by the kernel by their capabilities, and some
applications use local identi ers. The kernel provides an interface for attaching
1.5. The main abstractions of ChorusOS 7
resources to an actor. It is important to note that unlike a Unix process, a Cho-
rus actor does not execute anything. As we will see, the system distinguishes
between user and supervisor actors depending on whether they use a private
address space or share the kernel address space. Actors are presented in detail
in chapter 6;
 threads: a thread is the unit of execution and corresponds to a single ow of
sequential execution of a program. Threads are particular resources of actors:
when it is created, a thread is attached to an actor and will remain attached
to this actor during its entire lifetime. For example, a classic Unix process cor-
responds to an actor where a single thread is executing. A thread is identi ed
by a local identi er in the context of its actor. Thus, naming a thread requires
naming its actor and its local identi er in that actor.
An actor may contain an arbitrary number of threads, possibly none. Threads
may be dynamically created and deleted. Threads are presented in chapter 8;
 messages: messages are untyped sequences of bytes that can be exchanged
between threads;
 ports: these are the communication endpoints where messages are sent to
and received from by threads. Ports are resources of actors: at any time, a port
is attached to a single actor. Ports are the only resources that can migrate from
one actor to another. ChorusOS provides two semantics for communication via
ports: the communication may be synchronous (RPC mode) or asynchronous
(send/receive mode). Messages are sent by a thread from one source port to a
destination port. Ports are named by unique identi ers or by local identi ers
in the context of a particular actor;
 port groups: a port group is a set of ports. ChorusOS allows a message
to be sent from a port to a port group. ChorusOS provides various semantics
for sending a message to a port group: the message may be addressed to all
members of the port groups (broadcast addressing) or to one port of the group
(unicast or functional addressing) with di erent modes for the selection of the
destination port. Capabilities are used for naming port groups. These di erent
abstractions related to communication are described in chapter 13;
 memory regions: a memory region is a contiguous range of logical valid
addresses in the address space of an actor: the address space of an actor is thus
divided into non overlapping regions. Regions may be dynamically created and
deleted in an actor by threads. The services o ered for memory management
are presented in chapters 7 and 15.
These di erent abstractions and their relationships are summarized in the fol-
lowing gure:
8 Chapter 1. General presentation
message broadcast to the group G2
message sent in point to point mode

G1 port group G2

threads
regions
actor A1 actor A2 actor B1 an empty actor
site A site B
network
Chorus domain

where
- the Chorus domain consists of two sites named A and B where a ChorusOS is
running;
- the actor A1 belongs to the site A and encapsulates two ports, two memory
regions and two threads; the actor A2 on site A encapsulates two ports, three
regions and one thread. The actor B1 belongs to the site B and encapsulates
three ports, one region and three threads and nally, there is an empty actor
on site B (this actor just contains a port);
- two port groups are de ned: the group G1 contains three ports that are all
located on the site A and the group G2 contains one port located on the site A
and another one on the site B;
- one message is sent in one-to-one communication and another one is broadcast
to all ports of the group G2.

1.6 The architecture of ChorusOS


1.6.1 Overview
Depending on its con guration options, a given instance of the ChorusOS oper-
ating system consists of a kernel and di erent actors, called ChorusOS actors.
These di erent actors contribute to the implementation of the selected services.
The ChorusOS itself may be divided into two di erent layers, while a third layer
corresponds to the applications.
A typical con guration corresponds to the following picture, where the edges
are labelled with the names of some of the functions of the corresponding API.
1.6. The architecture of ChorusOS 9

Application
actorCreate
afexec
APPLICATION LAYER open
shm open await

commands: C INIT RDBC


mount afexec
route

ADMIN open
shm open AM
open
read

IOM SHM REP


ioctl
mount

OS LAYER
NUCLEUS LAYER
D mc146818 D i8254 PD KERN

drivers
Therefore, con guring a system consists mainly of de ning which actors will
be created when loading the system, and which modules in each actor will
be included: this process re ects both the con gurability and modularity of
ChorusOS.
1.6.2 The nucleus layer
This section describes the di erent components of this layer which provide the
basic services, as shown in the above picture.
1.6.2.1 The micro-kernel
The low-level basic services are provided by a core executive, micro-kernel cor-
responding to the KERN actor in the picture. It provides the services required
to support real-time applications: management of actors, threads, scheduling,
memory, synchronization, cross actor invocations (LAPs), and so on.
This executive supports multiple independent applications running either as
system applications (that is, sharing the address space of the kernel) or as user
applications. When building a given instance of the kernel, it is possible to
add separate modules providing speci c services meeting the needs of the ap-
plications intended to be executed on the system. These modules correspond
to optional features which may be selected when con guring the system (see
1.6.5 and 2.4).
10 Chapter 1. General presentation
1.6.2.2 The drivers
The nucleus layer also includes drivers which represent the hardware compo-
nents. These drivers are implemented as separate actors (for example, in the
picture the actors D i8254 and D mc146818): they allow the operating sys-
tem to interact with the hardware components. More information on drivers is
given in chapter 16.
1.6.2.3 The PD actor
Finally, this layer includes the PD actor which o ers a high level API (see
chapter 10) to manage per-thread or per-actor private data. One of the main
uses of these data is for the de nition of destruction handlers for actors. This
service is also used to retrieve the le context of extended actors.
1.6.3 The OS layer
1.6.3.1 Overview
This layer provides higher level services: management of input/output, network,
le systems, administration. It also o ers a high level management of actors. It
is composed of di erent specialized actors interacting with the nucleus layer.
Within ChorusOS 4.0, the architecture of the OS layer has evolved to satisfy
the following objectives:
 achieve a true modularity to enforce con gurability;
 enhance I/O performance;
 remove unused features.
The main aspects of the evolution of this architecture since the ChorusOS 3.2
release are the following:
 all cross actor invocations are now based on LAP invocations. The trap
mechanism is no longer used: this means that no binary compatibility with
previous releases of ChorusOS is provided;
 the AM and IOM actors are truly independent. Thus, a system may be
con gured with one or the other or both actors: services provided by each actor
are clearly identi ed. The AM is a regular client of the IOM: it uses standard
calls (open, read, . . . ) to access les. The IOM is more modular: di erent
independent features allow speci c services to be selected;
 a new embedded actor (ADMIN has been added to perform administrative tasks.
We now detail the various actors shown in the picture, which may or may not
be present in a given system, depending on the con guration that has been
adopted (and the system actors that have been embedded in the corresponding
archive).
1.6. The architecture of ChorusOS 11
1.6.3.2 The C INIT actor
It rst plays the role of a remote-shell daemon and listens for commands trans-
mitted by users via the rsh command. It also acts as a command interpreter.
This actor uses the services provided by the AM, the IOM and the ADMIN
actors. It is presented in detail in 3.1.
1.6.3.3 The IOM actor
This actor provides management of di erent input/output operations, depend-
ing on the related features that are selected. It has been upgraded with FreeBSD
2.2 source code. It includes le systems (UFS, NFS, MSDOS le system), dif-
ferent POSIX operations on les, sockets, message queues, shared memory or
pipes.
1.6.3.4 The AM actor
The actor manager is an optional component of the OS layer (corresponding
to the ACTOR EXTENDED MNGT feature) which serves calls related to actor man-
agement like afexec, akill and acreate. Its interface provides a high level
service for creating, loading, running and deleting actors. Actors that are cre-
ated by this speci c actor are commonly called c actors or . Unlike actors
created by directly invoking the kernel's service which are empty when created,
c actors look like classic POSIX processes. First, once created, a c actor has
a non empty address space: this address space gives access to a program and
to data which are loaded by the loader of the actor manager. A thread is also
created: it is called the main thread and it executes the main function of the
corresponding program. From the IOM point of view, this actor is a client like
other actors (except it is the only client of the IOM which requests le context
inheritance between actors). When an actor is created through an afexec call,
the AM opens the le to be loaded by invoking the IOM. Thus, a c actor has
speci c attributes, among them a le context that allows POSIX operations on
les.
1.6.3.5 The ADMIN actor
This mono-threaded actor was introduced in release 4 of the system. The role
of this actor is to embed all administration utilities which may be required to
administer a ChorusOS system when no le system is available. Depending on
the con guration of speci c features, it provides system calls such as ifconfig,
mount, rarp or route which are accessed through a lap's invocation.
12 Chapter 1. General presentation
Most of these services are also available as binary executable les which may
be accessed within the /bin directory of the exported tree.
1.6.3.6 The SHM REP actor
It provides a POSIX-compatible shared memory library (POSIX 1003.1). This
actor is created by the IOM actor when the POSIX SHM feature (which now
applies to the IOM) is active. The only function of this actor is to provide an
address space for shared memory segments provided by the feature. We will see
that this actor is an empty actor when created (see 7.8.1).

1.6.3.7 The RDBC actor


The Remote Debugger Actor Level Daemon (RDBC) o ers debugging
support for ChorusOS application actors and allows the use of debuggers like
gdb or xray. It corresponds to the DEBUG APPLI feature.

1.6.4 The application layer


This includes all actors developed by users. These actors use the services pro-
vided by the two layers above it through programming interfaces.

1.6.5 The di erent features


As already stated, a kernel is composed of an executive module and optional
modules which extend the minimum con guration. Speci c features also apply
to the OS layer. We now present the di erent features available, which are
identi ed by a symbolic name.
1.6.5.1 Actor management features
 ACTOR EXTENDED MNGT: this feature provides extended management functions
for actors. It provides dynamic loading and unloading of actors by using the
services of the AM and the IOM actors;
 USER MODE: this feature allows user actors to access the APIs exported by the
kernel modules corresponding to active features. When this feature is selected,
the VIRTUAL ADDRESS SPACE feature is also required;
 DYNAMIC LIB: this feature (it requires ACTOR EXTENDED MNGT) provides sup-
port for creating dynamic actors: these actors use dynamic libraries loading
which are mapped in the actor's address space at execution time (see 4.6);
1.6. The architecture of ChorusOS 13
 GZ FILE: this feature (which requires ACTOR EXTENDED MNGT) enables dynam-
ically loaded actors and dynamic libraries to be uncompressed at load time.
1.6.5.2 Scheduling option
As previously mentioned, the executive includes a scheduler which implements
a fo policy:
- all threads have a xed priority;
- the thread with the highest priority is elected;
- once elected, a thread releases the processor either when entering the sleeping
state, or when a thread with a higher priority is ready to run.
The ROUND ROBIN scheduler feature enables the de nition of a round robin pol-
icy. Within that policy, round robin time slicing is added. Therefore, threads
having the same priority share the processor according to the value of the con-
gurable time quantum.
1.6.5.3 The memory management features
Three di erent models of memory management may be selected through the
two features VIRTUAL ADDRESS SPACE and ON DEMAND PAGING:
 the MEM FLAT model is implemented in the executive (no speci c feature):
it corresponds to a at memory management where the kernel, the system ac-
tors and all the applications share a single at and unprotected address space
where logical addresses match physical addresses;
 the MEM PROTECTED model: it extends the MEM FLAT option by de n-
ing a per-actor protected address space for user actors. It provides a cheap way
of containing actor's faults. No swap or demand paging is provided. This model
corresponds to a con guration where the VIRTUAL ADDRESS SPACE memory fea-
ture is selected (and ON DEMAND PAGING is not);
 the MEM VIRTUAL model: it supports full virtual memory management
with demand paging and memory mapping. It also exports a generic inter-
face enabling implementation of distributed shared memory facilities across the
network: it requires the availability of at least one coherency mapper. The
corresponding module is added when selecting both memory features.
1.6.5.4 The time management features
The executive includes a basic one-shot timeout service. The following features,
which will be presented in chapters 12 and 14, are also available:
 DATE: maintains the time of day. Time, like in UNIX systems, is de ned as
the interval since the 1st January 1970;
14 Chapter 1. General presentation
 TIMER: provides a high-level timer service (one-shot or periodic timers) which
may be used for any type of actor;
 VTIMER: provides di erent functions for controlling and accounting thread
execution. These functions are usually used by subsystems (or personalities)
for de ning speci c timer services or limiting the CPU use of applications;
 PERF: this benchmark timing feature provides a very precise measurement of
short events;
 RTC: provides a real-time clock (for example on a PC by using the NC146818
circuit which integrates a CMOS-RAM memory).
1.6.5.5 The inter-thread synchronization features
The executive includes basic mechanisms: thread semaphores and mutexes that
will be presented in detail in chapter 9 together with the following speci c fea-
tures:
 SEM: provides general semaphores operations;
 EVENT: provides event ags sets management. A set is a group of bits in mem-
ory associated with a threads' waiting queue. Threads can wait on conjunctive
or disjunctive subsets of events;
 RTMUTEX: provides real-time mutexes based on threads' priority inversion to
avoid contention.
1.6.5.6 The local invocation optional features
Two features extend the basic service of local invocation (local access points)
that will be presented in chapter 11:
 LAPBIND: de nes a name binding service for getting descriptors of local access
points indirectly;
 LAPSAFE: does not provide an interface, but it modi es the semantics of cre-
ation and invocation of local access points, in particular by providing a safe
shutdown of service routines.
1.6.5.7 The communication optional features
The following options allow threads, possibly belonging to di erent actors, to
communicate even if they do not share a common memory space:
 MIPC: provides copy free message communication between di erent actors
through a shared communication environment (mailboxes);
 IPC: provides services for communication using messages on a single node.
Communication is achieved through ports and the following types of communi-
cation are supported:
1.6. The architecture of ChorusOS 15
- asynchronous communication;
- synchronous communication (RPC mode);
- multi-cast communication;
 IPC REMOTE: extends the IPC feature and provides location-transparent com-
munications between actors belonging to di erent interconnected sites. When
this feature is active, the IPC REMOTE COMM allows the speci cation of a com-
munication method;
 POSIX MQ: provides a POSIX-compatible real-time message queue library
(POSIX 1003.1). This feature applies to the IOM actor;
 POSIX SHM: provides a POSIX real-time shared memory API.
1.6.5.8 The le system optional features
The following options are provided:
 UFS: provides support for local Unix File System (the interfaces supported are
SCSI, IDE or RAM);
 MSDOSFS: provides support for local MSDOS le systems (the interfaces sup-
ported are SCSI, IDE, RAM and Flash memory);
 FIFOFS: this feature provides the support for named pipes. It requires both
POSIX SOCKETS and AF LOCAL features and either NFS CLIENT or UFS;
 NFS CLIENT: provides POSIX compatible system calls for input/output oper-
ations on top of the NFS le system. It only covers the client side of the NFS
protocol, which means that the host system where the le system is located
must provide the server side of the protocol. This feature may be used on top
of Ethernet, ppp or slip protocols and requires the POSIX SOCKETS feature;
 NFS SERVER: implements the server side of the NFS protocol. Thus, it al-
lows remote systems to access local les, if any. This feature requires the
POSIX SOCKETS feature and may be con gured for ufs or msdos le systems
(corresponding to the UFS or MSDOSFS features).
1.6.5.9 The debugging features
 LOG: provides support for logging activity on a target system;
 DEBUG SYSTEM: allows remote debugging of the ChorusOS system using the
Microtec XRAY debugger;
 DEBUG APPLI: allows remote debugging of ChorusOS applications using the
Microtec XRAY debugger, which communicates with a speci c actor named
RDBC on the target system.
16 Chapter 1. General presentation
1.6.5.10 The C INIT optional feature
 LOCAL CONSOLE: this feature gives access to the C INIT commands through
the local console of the target. Within the feature, a thread is started in the
C INIT actor to interpret commands entered from the console;
 RSH: when this feature is enabled, it is possible to access C INIT commands
by using the remote-shell service (rsh).
1.6.5.11 The I/O management options
Some of the features presented here rely on the support o ered by the BSP to
access the bus.
 DEV MEM: provides a raw interface to memory devices (/dev/zero, /dev/null,
/dev/kmem and /dev/mem);
 BPF: provides a raw interface to data link layers in a protocol independent
fashion;
 IDE DISK: provides an interface to access IDE local disks;
 SCSI DISK: provides an interface to access SCSI local disks;
 SCSI NCR53C8XX: provides a driver for SCSI disks;
 SCSI CDROM: provides an interface to access SCSI CDROMs;
 RAM DISK: provides support for implementing a le system in memory;
 FLASH: provides support for ash media;
 VTTY: provides support for serial line on top of the BSP driver for high-level
protocols. It is used by the SLIP and PPP features.
1.6.5.12 The network features
 SLIP: allows the use of serial lines as network interfaces with the serial line
internet protocol;
 PPP: allows the use of serial lines as network interfaces with the point-to-point
protocol;
 POSIX SOCKETS: provides a POSIX-compliant (1003.1g) interface with socket
calls. The service is limited to sockets in the AF INET domain;
 AF LOCAL: extends the socket service provided by the previous feature to the
AF LOCAL domain.

1.6.5.13 The administration features


These features allow speci c modules providing corresponding services and their
interfaces be to included in the code executed by the ADMIN actor. These mod-
1.6. The architecture of ChorusOS 17
ules are invoked by the C INIT actor; whenever the corresponding feature is
not enabled, the built-in command of the C INIT actor returns an error;
 ADMIN CHORUSSTAT: corresponds to the built-in chorusStat C INIT's com-
mand: this service allows a user to get statistical information about the Cho-
rusOS system (see 3.5.2);
 ADMIN IFCONFIG: provides support for the built-in ifconfig command of
C INIT;
 ADMIN MOUNT: provides support for the built-in mount and umount commands
of C INIT. When it is enabled, UFS, MSDOS and NFS le systems may be
dynamically mounted. It is therefore possible to run a command dynamically
loaded from a mounted le system;
 ADMIN RARP: provides support for the built-in rarp command of C INIT;
 ADMIN SHUTDOWN: provides support for the built-in shutdown command of
C INIT;
 ADMIN NETSTAT: provides support for the built-in netstat command of C INIT;
 ADMIN ROUTE: provides support for the built-in route command of C INIT.
1.6.6 The restart option
The RESTART feature provides fast recovery after a failure (high restart service
is available at di erent levels: site, actor, group of actors running on the same
site). This book does not cover the services of this feature.
Chapter 2
Building and booting a
ChorusOS system
2.1 Overview
In this chapter, we describe how standard versions of the system may be built
and installed on a host system once the delivery has been retrieved. We then
describe the di erent tools which allow a speci c ChorusOS system image to
be built. This system image contains a con gured version of the ChorusOS
system, loadable on a speci c target. It includes at least a nucleus (or kernel)
and possibly system actors (like the AM or the IOM actor), called ChorusOS
actors, which contribute to the implementation of selected features. The system
image may also include some speci c user applications which will be loaded and
started at boot time. Finally, we show how this type of system image may be
loaded and booted on a target machine. We assume here that the host system
is a SPARC platform under Solaris 2.6 or Solaris 7 and that the target system
is an ix86 based PC.
2.2 Installing the delivery
2.2.1 Downloading the les
First, the les have to be downloaded, using standard Internet tools, and using
the URL, the login name and the password provided by SUN. The various les
will be installed in an empty directory with the symbolic name download dir; in
our example, it will be $HOME/CHORUS/r4. This operation requires at least 100
Mega bytes free disk space (this requirement is speci ed in the README.html
le on the URL). Once this operation is complete, the contents of the directory
are the following:
19
20 Chapter 2. Building and booting a ChorusOS system
--> cd $HOME/CHORUS/r4
--> ls
chorus-doc.zip extract.sh SUNWewmab.zip xray.zip
chorus-x86.zip SUNWewcab.zip SUNWewsab.zip
-->

2.2.2 Installing the packages


Di erent packages may now be installed by running the extract.sh le. It
is a shell script which unzips the les and runs an installation wizard which
uses standard Solaris tools to install packages (pkgadd utility). Thus, the root
password of the host system must be provided when the wizard is executed.
The les are installed in a directory with the symbolical name install dir (its
default value is /opt/SUNWconn/SEW).
--> cd $HOME/CHORUS/r4 (= extract dir
--> sh extract.sh (= execute the script shell
.......
root passwd: (= must provide root password
.......
-->

2.2.3 Organization of the installed les tree


2.2.3.1 The rst level
Once the installation has been completed and after exiting the Web start Wiz-
ard, an installation log le may be consulted to check whether or not the in-
stallation was successful.
In the next sequence, we still use the ls standard Unix command to list the
various les that have been installed:
--> ls -R /opt/SUNWconn/SEW (= get the list of installed les
:/opt/SUNWconn/SEW
4.0/

/opt/SUNWconn/SEW/4.0:
chorus-doc/
chorus-x86/

/opt/SUNWconn/SEW/4.0/chorus-doc:
html/
man/
pdf/
ps/

..... [ more lines ] ......


-->
2.2. Installing the delivery 21
The next gure describes the rst levels of the tree corresponding to the general
organization of these les: install dir

4.0

chorus-doc/ chorus-x86/ readme.html

man/ pdf/ kernel/ opt/ os/ src/ tools/

XRAY/

 the chorus-doc subdirectory gives access to the on-line documentation. The


man subdirectory contains the manuals of the di erent commands (section 1)
which may be accessed on the host system using the man command. The
MANPATH variable of the shell environment must be modi ed to access this man-
ual with the following command for sh, ksh or bash shells
--> export MANPATH=install dir/4.0/chorus-doc/man:$MANPATH
or by the following command for csh or tcsh shells:
--> setenv MANPATH install dir/4.0/chorus-doc/man:${MANPATH}
The pdf subdirectory contains the documents that may be consulted using a
browser.
 the chorus-x86 subdirectory is the root of a tree which contains the di erent
les used to build a system: its organization re ects the architecture of the
ChorusOS system. It further contains the following subdirectories:
- tools: contains the tools used to build a system archive on the host
system;
- kernel: contains les used to build the kernel;
- os: contains les that are used to build the os layer;
- src: contains a number of source les used to build a system image;
- opt: contains les used to build optional modules of a system image.
2.2.3.2 The tools directory
The tools directory is the root of a subtree containing the commands provided
to build, on the host system, a system image for the selected target. These
commands can then be used to develop applications to be launched on the
target system once the archive has been loaded and booted on it.
22 Chapter 2. Building and booting a ChorusOS system
This subtree has the following structure:

chorus-x86/

tools/

ChorusOSMkMf configure ews/ host/ ix86/

bin/ etc/ include/ lib/ solaris/

configure ews
5.0/

bin/

gcc cross development


tools

The PATH variable of the shell environment should be modi ed by adding the
directory containing the various utilities that will be used on the host system:
--> export PATH=install dir/4.0/chorus-x86/tools/host/bin
-->

2.2.3.3 The kernel directory


This directory contains the con guration les (expressed in XML) and object
les needed to build standard versions of the nucleus. The corresponding subtree
has the following structure:
kernel/

bin/ conf/ include/ obj/

PD.r PROF.r basic extended mini mkconfig/ mkimage/ chorus/ kern/

xml description les exec/ ipc/ mem/


2.2. Installing the delivery 23
2.2.3.4 The os directory
This directory contains the con guration les (expressed in XML) and object
les needed to build standard versions of the operating system. The correspond-
ing subtree has the following structure:
os/

bin/ conf/ obj/ root/

mkconfig/ os/ bin/ etc/

xml description admin/ am/ cinit/ arp.r cs.r ftp.r


les for
AM, C INIT, ADMIN

It is important to note that in release 4.0 the code corresponding to the IOM
actor is provided as source code and thus does not appear in this tree.

2.2.3.5 The src directory


This directory contains con guration les and a number of C source les used
to build standard con gurations. These les relate to the description of the
nucleus layer (source code adapted to board support: processors and drivers)
and the description of the OS layer (source code of the IOM). The corresponding
subtree has the following structure:
src/

iom/ nucleus/

conf/ src/ bsp/

mkconfig/ iom/ drv/ x86/

xml les C source les src/ i386at/

C source les C source les


24 Chapter 2. Building and booting a ChorusOS system
2.3 Building a standard image
2.3.1 Prerequisites
We rst assume that the packages have been installed correctly as described
in the previous section. The next operations should not be performed as root
user. In the example, we will perform them as jmr user. The environment
(PATH and MANPATH variables) must have been correctly de ned as speci ed in
the previous section. We can check that this is the case simply by calling the
configure utility (without argument) and the man command for this utility:
--> configure
Usage:
configure [-f profile-file...] [-s source-dir...] [-b bin-dir...]
[-t build-dir...] [-d name=definition...]

Examples:
configure -b ChorusDir -d SILENT=
configure -b ChorusDir -s SourceDrv SourceBsp
--> man configure

NAME
configure - prepare a build directory for ChorusOS
.......... [more lines] ..........
-->

We intend here to build a rst system image corresponding to a prede ned


standard pro le. To that end, we rst create a new directory where this image
will be built and change to that directory; this directory will have the symbolic
name build dir.
--> mkdir $HOME/CHORUS/r4/build
--> cd $HOME/CHORUS/r4/build
-->

2.3.2 The configure utility


This command is used to prepare a build directory for ChorusOS. It allows
you to select di erent components either in source or in binary code from the
installed les tree and prepares a directory where these di erent ChorusOS com-
ponents will be built. The actions taken to build these di erent components
may di er from one component to another; the actions may be compiling, build-
ing a ChorusOS archive, or con guring or building a NFS root directory that
will be mounted on a ChorusOS target.
As result of the command, the build dir directory will contain two les:
 Makefile containing the rules for building the system image;
 Paths containing the path settings used to build the system image.
2.3. Building a standard image 25
The general form of the command is the following:
[ pro le- le ...] [-s source-dir ...] [-b bin-dir ...] [-d var=value ...] [-t build-dir]
configure -f
where
 pro le- le is used for source components of the nucleus;
 source-dir corresponds to the pathname of a source component;
 bin-dir corresponds to the pathname of a binary component;
 var=value adds the de nition of the corresponding variable in the Paths con-
guration le;
 build-dir is the pathname of the build directory (by default it is the current
directory).
In the following sequence, we use the configure utility to create the build envi-
ronment corresponding to the packages we installed in the previous section. The
di erent components (source and binary les) are all located in the tree whose
root is install dir/4.0/chorus-x86. Thus, we rst de ned a DIR environment
variable corresponding to that directory:
--> cd $HOME/CHORUS/r4/build
--> DIR=/opt/SUNWconn/SEW/4.0/chorus-x86
--> configure -b $DIR/kernel $DIR/os $DIR/tools -s $DIR/src/nucleus/bsp/x86\
$DIR/src/nucleus/bsp/drv $DIR/src/nucleus/bsp/x86/i386at $DIR/src/iom
-->

We can check the contents of the build directory:


--> ls $HOME/CHORUS/r4/build
Ident Makefile Paths
-->

2.3.3 Building the system image


We can now create a standard system image by calling the make command. If
the chorus argument is passed as argument, a complete system image is built
(a kernel-only system, that is without system actors, may be built by passing
kernonly as argument):
--> cd $HOME/CHORUS/r4/build; make chorus
.......... [many lines] .......
Start mkimage
Brief log file: /home/jmr/CHORUS/r4/build/image/bmon/chorus/log.brief
Verbose log file: /home/jmr/CHORUS/r4/build/image/bmon/chorus/log.verbose
Layout file: home/jmr/CHORUS/r4/build/image/bmon/chorus/layout.xml
Image file: /home/jmr/CHORUS/r4/build/chorus.bmon
Finish mkimage
-->
26 Chapter 2. Building and booting a ChorusOS system
As a result of the command, the contents of the build dir directory are the fol-
lowing:
--> ls -F $HOME/CHORUS/r4/build
build-BSP/ build-IOM/ Ident obj/
build-DRV/ chorus.bmon* image/ Paths
build-DRV_F/ conf/ Makefile
-->

The chorus.bmon contains a standard system image that can be used directly
or be modi ed to build new ones, for example by adding or removing features or
modifying tunable parameters with con guration tools (ews or configurator)
that will be presented later. This standard image has the following character-
istics:
 list of system actors: ADMIN, AM, C INIT, and IOM;
 list of kernel features: DATE, EVENT, IPC, LAPBIND, LAPSAFE, LOG, MIPC, PERF,
RTC, SEM, TIMER, USER MODE, VIRTUAL ADDRESS SPACE;
 list of AM features: ACTOR EXTENDED MNGT;
 list of C INIT features: RSH;
 list of IOM features: AF LOCAL, BPF, DEV MEM MSDOSF, NFS CLIENT, POSIX SOC-
KETS, RAM DISK;
 list of ADMIN features: ADMIN IFCONFIG, ADMIN MOUNT, ADMIN RARP, ADMIN
ROUTE, ADMIN SHUTDOWN.
The four directories: build-BSP, build-DRV, build-DRV F and build-IOM con-
tain the binary les generated from the sources les (drivers, bsp and iom).
The obj directory contains the binaries that have been built. The correspond-
ing tree lists the les kern.r, ADMIN.r, AM.r, C INIT.r, and so on.
The image directory contains the object les generated during the process of
generation.
Finally, the conf directory is the root of a subtree which contain all the con-
guration les (expressed in XML); these les contain the characteristics of the
system image. These les may be modi ed with standard tools to prepare the
production of a new system image. We present these les in section 2.4.1 when
describing how a system image may con gured.

2.3.4 Building the root le system


We now build a le tree which is intended to be NFS mounted on target sys-
tems running the ChorusOS; it contains various binary les executing standard
commands and administration les.
2.3. Building a standard image 27
The root of this tree is the root subdirectory of the build dir directory and it
is built simply by invoking the make command:
--> cd $HOME/CHORUS/r4/build
--> make root
...... [more lines] .......
-->
After this operation, the contents of the root directory correspond to the
following tree:
root/

bin/ image/ dev/ lib/ etc/ Makefile

arp chorusStat cp cs ls fstab nfsd services

a) the bin subdirectory contains a number of binary les which can be executed
on a target system (we will see how in the next chapter):
 basic Unix like commands: arp, chmod, cp, date, dd, df, format, fsck, ls,
mkdir, mkfifo, mknod, mount, mv, newfs, pax, rm, rmdir, touch, umount;
 network commands: arp, domainname, ftp, hostname, ifconfig, inetNSdns,
inetNShost, inetNSnis, netstat, nfsstat, pppstart, route, shutdown, uname,
ypbind, ypcat, ypmatch, ypwhich;
 speci c ChorusOS commands:
- chat: automated conversational script with a modem;
- chorusNSinet, chorusNSsite: ChorusOS name servers;
- chorusStat: information about ChorusOS resources;
- cs: report ChorusOS resource status (see 3.8.3);
- dhclient: Dynamic Host Con guration Protocol client;
- disklabel: disk pack label;
- flashdefrag: defragment a ash memory;
- fsck dos: create an MS-DOS (FAT) le system;
- newfs dos: check and repair an MS-DOS (FAT) le system;
- profctl: ChorusOS pro ling control tool;
- rdbc: ChorusOS remote debugging daemon;
- slattach: attach serial lines as network interfaces;
28 Chapter 2. Building and booting a ChorusOS system
- sysctl: get or set kernel state;
- tclsh: interpreter for Tcl, an embeddable scripting language;
b) the etc subdirectory contains:
 traditional administration les like exports, protocols or services;
 binary executable for servers like ftpd, nfsd or portmap;
 and speci c ChorusOS administration les like security or sysadm.ini (see
3.1).

2.4 Con guring and tuning a system image


2.4.1 Overview
As mentioned previously, di erent types of con guration options are available:
 features;
 static tunable parameters;
 dynamic tunable parameters de ning the environment;
 system image components and applications which are loaded at boot time.
The values of these con guration parameters are registered in XML con gura-
tion les that may be modi ed using various tools.
2.4.2 The con guration les
The di erent levels of con guration correspond to di erent con guration les
which are located in the conf directory of the merge tree. The corresponding
tree has the following structure:
conf/

ChorusOS.xml mkconfig/ basic extended mkimage/

kern.xml admin.xml am.xml .......... iom.xml mkimage.xml family.xml ..........


applications.xml

where
 the ChorusOS.xml le is the top level con guration le. The whole con gu-
ration of a system can be accessed through this le. It contains references to
the other con guration les;
 the mkconfig subdirectory is a directory which contains the description of
the di erent components of the systems:
2.4. Con guring and tuning a system image 29
- the kern.xml le contains the de nition of the features of the kernel and
their dependencies, the values of the tunable parameters of the kernel and
the default values for a standard con guration;
- the kern action.xml, kern action f.xml and kern f.xml les contain
speci c con guration actions and production rules used for the con gura-
tion;
- the nucleus.xml, os.xml, admin.xml, am.xml, cinit.xml, iom.xml les
correspond to the de nition of the corresponding components of the sys-
tem. Action les like am action.xml or iom action.xml contain con gu-
ration actions for the corresponding components;
 the mkimage subdirectory gives access to the les containing the information
needed to build a given system image. It contains the following les:
- mkimage.xml: contains two con gurable declarations:
 BOOT MODE which may be ram to build an image for RAM, or rom to
build an image for ROM;
 SYSTEM which may be set to chorus, kernonly or kts depending on
whether the system image built will contain a default chorus system
image, a kernel only system image or will include kernel tests;
- family.xml: contains the family dependent de nitions;
- target.xml: contains the binary of all the con guration options related
to the BSP and the list of the drivers;
- model.xml: contains the binary models for executable les;
- system.xml: contains all system binaries and the con guration of the
system image;
- applications.xml: contains the description of all applications that will
be included in the system image;
 the basic and extended les contain the de nitions of the prede ned con g-
uration pro les. They correspond to the following:
- the basic pro le: this is a con guration providing limited services in order
to keep the footprint small. Applications are generally embedded within
the system image and launched either at boot time or later from a local
le system. It uses the MEM FLAT memory management model. Adminis-
tration is purely local: the C INIT actor is directly accessed through the
system console. It includes the following features: ACTOR EXTENDED MNGT,
ADMIN IFCONFIG, ADMIN MOUNT, ADMIN ROUTE, ADMIN SHUTDOWN, AF LOCAL,
DATE, EVENT, IPC, LAPBIND, LAPSAFE, LOCAL CONSOLE, LOG, MIPC, MSDOSFS,
PERF, POSIX SOCKETS RAM DISK, RTC, SEM, TIMER, USER MODE;
30 Chapter 2. Building and booting a ChorusOS system
- the extended pro le: provides more general services. It uses the MEM PRO-
TECTED memory management model and includes support for networking
and an NFS client allowing mounting of remote les. It includes the fol-
lowing features: ACTOR EXTENDED MNGT, ADMIN IFCONFIG, ADMIN MOUNT,
ADMIN RARP, ADMIN ROUTE, ADMIN SHUTDOWN, AF LOCAL, BPF, DATE, DEV
MEM, EVENT, IPC, LAPBIND, LAPSAFE, LOG, MIPC, MON, MSDOSFS, NFS CLIENT,
PERF, POSIX SOCKETS RAM DISK, RSH, RTC, SEM, TIMER, USER MODE, VIR-
TUAL ADDRESS SPACE.

2.4.3 Con guration tools


Two di erent con guration tools are provided:
 a graphical tool: ews
 a command-line tool: configurator
These two tools act on an XML con guration le, typically conf/ChorusOS.xml.
2.4.3.1 The graphical tool ews
To use this tool, the SUN Java JDK 1.2 is required. The utility is called as
ews [-c con g le]
All the operations allowing con guration and tuning of a ChorusOS system may
be achieved by using the various menus provided by the navigator tree. Once
the con guration phase is completed, the system image corresponding to the
resulting con guration le may be built by selecting the ChorusOS Con gura-
tion item in the tree and then using the build item; the con guration le must
have been saved and need to be valid in order to build the image.
2.4.3.2 Command-line tool: configurator
The general form of this command is
configurator -c con g le command
where con g le (typically conf/ChorusOS.xml) corresponds to the con gura-
tion le.
a) The help command
Gives on-line help on how to use the command.
b) Checking a con guration: check
The check command checks the con guration.
c) Displaying a con guration in HTML: display
By calling the command
configurator -c con g le -display le
2.4. Con guring and tuning a system image 31
the HTML le corresponding to the given con guration le is built; this le
may then be displayed using a browser.
d) Selecting a con guration pro le
Two prede ned con guration pro les are provided and may be selected to build
a system image with one of the two commands, depending on whether a basic
or a extended pro le is required:
configurator -c con g le -p conf/basic
configurator -c con g le -p conf/extended
e) Listing, adding or removing features
A feature has an ocial name and a value (true or false) in a given con gu-
ration le, the value true means that the feature is selected.
It is possible to check the value of a given feature whose name is feature name
or the values of all features by calling one of these two commands:
configurator -c con g le -list pattern
configurator -c con g le -list feature
The feature option returns the list of all features and the pattern option re-
turns the list of features whose names match the pattern.
Here is an example of a sequence using this utility to get the status of the
di erent features for the standard system image we built:
--> configurator -c conf/ChorusOS.xml -list feature
HOT_RESTART bool 'false'
USER_MODE bool 'true'
ROUND_ROBIN bool 'false'
VIRTUAL_ADDRESS_SPACE bool 'true'
ON_DEMAND_PAGING bool 'false'
MONITOR bool 'false'
SEM bool 'true'
EVENT:bool='true'
RTMUTEX:bool='false'
... [more lines] .........
NFS_SERVER:bool='false'
POSIX_MQ bool 'false'
POSIX_SHM bool 'false'
AF_LOCAL bool 'true'
BPF bool 'true'
POSIX_SOCKETS bool 'true'
PPP bool 'false'
SLIP bool 'false'
DEBUG_SYSTEM bool 'true'
-->
32 Chapter 2. Building and booting a ChorusOS system
A given feature may be added by calling the command
configurator -c con g le -set feature name=true
For instance, the ROUND ROBIN feature is added in the next sequence
--> configurator -c conf/ChorusOS.xml -set ROUND_ROBIN=true
-->
To remove a feature, its status must be set to false by using the command
configurator -c con g le -set feature name=false
For example, the next sequence removes the ROUND ROBIN feature in the con g-
uration le conf/ChorusOS.xml.
--> configurator -c conf/ChorusOS.xml -set ROUND_ROBIN=false
-->
Finally, the default value of a feature for a given pro le may be reset by calling
configurator -c con g le -reset feature name
After the call, the default value of the feature for the selected pro le is set.
f) Listing or changing tunable parameters
The tunable parameters are named by symbolic names, these names may include
the dot character (".") to re ect the hierarchical organization of the systems.
For example, kern.exec.maxThreadNumber identi es the maximum number of
threads managed by the executive of the kernels, am.afexecschedprio shows
the priority of the main thread of an actor spawned by calling the afexec service
of the actor manager. Each of these parameters has a default value.
The current values of all tunable parameters or speci c tunable parameters in
a con guration le may be retrieved by calling
configurator -c con g le -list tunable
configurator -c con g le -list tunable pattern
In the next sequence, the list of all tunable parameters is rst displayed; the
second request displays all tunable parameters speci c to the C INIT actor by
using the cinit.* pattern:
--> configurator -c conf/ChorusOS.xml -list tunable
kern.exec.maxCpuNumber int '1'
kern.exec.maxActorNumber int '64'
kern.exec.maxThreadNumber int '128'
kern.exec.dflSysStackSize int '0x3000'
......
kern.ipc.maxSiteNumber int '4'
......
iom.nfs.rsize int '8192'
......
am.afexecschedprio int '140'
2.4. Con guring and tuning a system image 33
am.afexec.userstacksize int '0x4000'
......
cinit.intrthreadprio int '30'
......
iom.nfile int '64'
......
--> configurator -c conf/ChorusOS.xml -list tunable cinit.*
cinit.intrthreadprio int '30'
cinit.defaultUid int '0'
cinit.defaultGid int '0'
-->

The value of a given parameter may be set to a given value or reset to its default
by calling the commands
configurator -c con g le -set name=value
configurator -c con g le -reset name
In the next sequence, we change the values of certain parameters:
--> configurator -c conf/ChorusOS.xml -set kern.exec.maxActorNumber=92
--> configurator -c conf/ChorusOS.xml -set kern.exec.maxThreadNumber=256
--> configurator -c conf/ChorusOS.xml -set am.afexecschedprio=100
-->
More precisely:
- the maximum number of actors is set to 92;
- the maximum number of threads is set to 256;
- the priority of the main thread of an actor dynamically created by the actor
manager is set to 100.
g) Listing or changing the system environment
The system environment is de ned by (name, value) pairs, where both name
and value are character strings. This environment is accessed by using the
commands
configurator -c con g le -list env
configurator -c con g le -list env pattern
configurator -c con g le -setenv name=value
configurator -c con g le -resetenv name
The environment contains an internal variable OS CONF whose value should not
be modi ed. Other variables it may be useful to de ne are ROOT HOSTNAME,
ROOT HOSTADDR and ROOT PATHNAME; they are used by the C INIT actor to mount
a root le system automatically.
--> configurator -c conf/ChorusOS.xml -list env
OS_CONF=:IOM:AM:ADMIN:C_INIT
-->
34 Chapter 2. Building and booting a ChorusOS system
The next sequence de nes di erent variables in the environment of the archive:
--> configurator -c conf/ChorusOS.xml -setenv ROOT_HOSTADDR=134.157.168.9
--> configurator -c conf/ChorusOS.xml -setenv ROOT_HOSTNAME=fluorine
-->

h) Embedding a user actor in the system image


We assume here that loadable supervisor code for an embedded actor has been
produced by the standard tools that we will present in the next chapter (cre-
ation of an Imakefile using the EmbeddedSupActorTarget and the command
ChorusOSMkMf), and that the corresponding le's absolute pathname is path-
name directory/embeddedActor_s.r.
The easiest way to add the actor to the system archive is to use the ews graphical
tool. The actor can also be added by modifying the contents of the con guration
le conf/mkimage/applications.xml which contains the list of applications
that will be included in the archive when building it.
i) Building or rebuilding the system archive
Once the con guration le has been modi ed, the archive may be built or re-
built by using the make command:
 a kernel-only system is built using the sequence
--> make kernonly
.........
 a complete ChorusOS system is built using the sequence
--> make chorus
.........

 the system is rebuilt using the sequence


--> make build
.........

2.5 Booting the ChorusOS system


The target will generally be rst started with a diskette containing a minimal
system which provides the Ethernet driver of the board and a number of net-
work protocols allowing the ChorusOS system to be loaded from a host system
through the network. Di erent host systems may be involved in that procedure;
we will assume here that only one Solaris host system provides all the services
required to boot ix86 based targets.
2.5. Booting the ChorusOS system 35
2.5.1 Preparing the boot monitor
Once it has been built, the system image has to be loaded and started on the
target system. The bootMonitor utility is used for that purpose; it uses stan-
dard network protocols (RARP and tftp) over an Ethernet network to load and
boot a ChorusOS archive. This utility is embedded in a ChorusOS archive
which contains a light version of the ChorusOS system (it includes all drivers
needed to communicate over an Ethernet network). This light version of Cho-
rusOS corresponds to the mini pro le (de ned in the conf subdirectory of the
build directory). The archive containing the boot monitor is generally put on
a diskette or into ash memory.
We now describe the di erent steps leading to the creation of this small archive
and how it may be written to a diskette. We assume here that we are using a
Solaris host station and that targets are based on ix86 processors.
a) con gure the Ethernet adapters
If PCI adapters are used, the next are not necessary. The configurator tool is
used to set the value of variables associated with the di erent adapters available
on the target system:
configurator -setenv ETHER ident=name,irq,io base[,mem base]
where:
- ident is a decimal value in the range [0-9] (this means that up to 10 adapters
can be declared);
- name is the type of the device (it may be SMC, NE2000 or EL3);
- irq is a decimal value corresponding to the interrupt request level on the bus;
- io base is the base for I/O ports expressed as a hexadecimal number;
- mem base is a hexadecimal value corresponding to the base for shared memory.
This parameter is only required for SMC adapters.
b) build the boot monitor archive
First, the mini pro le and the LILO loader are selected, Then the bootMonitor
image is built:
--> cd $BUILD_DIR
--> configurator -p conf/mini (= select mini pro le
--> configurator -set LOADER=lilo (= select lilo loader
--> make bootMonitor (= build the boot monitor image
.................................
-->

Once the boot monitor image is built (the le is bootMonitor.image), we re-


store previous selections (boot monitor loader [bmon] and standard pro le [for
example extended]):
36 Chapter 2. Building and booting a ChorusOS system
--> configurator -p conf/extended
--> configurator -set LOADER=bmon
-->
c) make the boot monitor diskette
The bootMonitor.image le is copied onto the diskette on the Solaris host
system. The vold daemon, which manages CD-ROM and oppy devices, must
rst be stopped by executing the /etc/init.d script as super-user. A diskette
may be formatted using the fdformat utility, and the le is copied with the
standard cp command (the dd utility may also be used). Finally, the vold
daemon is restarted:
--> su
Password: (= must provide root password
# /etc/init.d/volmgt stop (= stop vold daemon
# exit
--> fdformat -v /dev/fd0 (= format diskette
.................
--> cp bootMonitor.image /dev/fd0 (= copy bootMonitor onto diskette
--> su
Password: (= must provide root password
# /etc/init.d/volmgt start (= restart vold daemon
volume management starting.
# exit
-->

2.5.2 Setting up booting services on the host station


2.5.3 The RARP service
The rst information that the target system needs is its IP address; when it
is booted, it will send a RARP request to get it. Therefore, a RARP server
recognizing the target must be available. On that machine, the /etc/ethers
le is used as local source information about the Ethernet addresses of systems
on the Internet. One line of that le has the following format:
Ethernet address ocial hostname [# comments]
where the Ethernet address is expressed in standard form x1:x2:x3:x4:x5:x6
where xi is the hexadecimal value of a byte (that is between 00 and ff).
In our con guration, on the fluorine host system that we use for booting, de-
veloping and launching applications, information about di erent targets (carbon
and neon, for instance) is:
--> cat /etc/ethers
00:00:e8:e3:20:45 neon # Internet address 869DA80A [134.157.168.10]
00:60:97:c0:64:ed carbon # Internet address 869DA806 [134.157.168.6]
-->
2.5. Booting the ChorusOS system 37
The RARP daemon must be started. On a Solaris system, it would be done by
executing the following sequence:
--> su
Password: (= become super-user
# /usr/sbin/in.rarpd -a (= start rarp daemon
# exit
-->

2.5.4 The TFTP service


In a second step, the target will send a tftp request to the RARP server to get a
system image. Therefore, the tftp service must be available on the host system:
a) as root user create a /tftpboot directory if it does not already exist. This
directory will be searched to get the archive;
b) in the /etc/inetd.conf le, register the service as an active one by removing
the comment character #:
tftp dgram udp wait root /usr/sbin/in.tftpd in.tftpd -s /tftpboot
c) reinitialize the inetd daemon by sending the SIGHUP signal:
--> ps -ef | grep inetd (=
get pid of inetd
root 156 1 0 08:35:08 ? 0:00 /usr/sbin/inetd -s
--> su (= become super-user
Password
# kill -1 156 (= send SIGHUP to inetd
# exit
-->
d) the target will rst try to obtain from the host system a le whose pathname
is /tftpboot/IP address.ChorusOS.4.0, where IP address is the IP address the
target got from the RARP reply. This address is expressed as 8 hexadecimal
digits. For instance, the neon machine, whose IP address is 134.157.168.10,
will rst try to load a 869DA80A.ChorusOS.4.0 le from the /tftpboot direc-
tory. If this le exists, it should contain a bootable system archive.
If that le does not exist, the target will try to transfer /tftpboot/IP address.
This le should contain the IP address of the server where the bootable archive
is located (it may be a di erent one) and the name of the boot archive on that
server. For example on the fluorine server whose IP address is 134.157.168.9,
the /tftpboot/869DA806 le corresponds to the carbon system and contains
the following:
--> cat /tftpboot/869DA806
AUTOBOOT=YES
BOOTSERVER=134.157.168.9 (= address of the server
BOOTFILE=chorus_vm (= name of the archive
-->
38 Chapter 2. Building and booting a ChorusOS system
The /tftpboot directory must also contain the bootable archive chorus vm, it
should be there as the pathname of the archive is not an absolute one.

2.5.5 Connecting the console and booting


The target is connected to the host system via a serial line. On the host side,
the standard tip command is used.
We rst boot a system on the neon machine, once the diskette has been read,
we get the following on the console:
RAM size: 0x1000000 bytes

ChorusOS r4.0.0 for Intel x86 - Intel x86 PC/AT


Copyright (c) 1999 Sun Microsystems, Inc. All rights reserved.

Kernel modules : CORE SCHED_FIFO MEM_FLM .........


/pci/i8259: sun:pci-i8259-pic driver started
[
.......... more lines ]..........
Boot Monitor Loader (v1.0)

Searching for adapters...


Unit: 0 device name: pci10ec,8029@6,0

Using unit 0

My IP 134.157.168.10, RARP Server IP 134.157.168.9

Loading file 869DA80A.ChorusOS.4.0 on server 134.157.168.9: loaded!


[
.......... more lines ]..........
Booting downloaded file.
Boot new image ...
RAM size: 0x1000000 bytes

ChorusOS r4.0.0 for Intel x86 - Intel x86 PC/AT


Copyright (c) 1999 Sun Microsystems, Inc. All rights reserved.

Kernel modules : CORE SCHED_FIFO SEM MIPC IPC_L MEM_VM KDB TICK MON ENV
ETIMER LOG LAPSAFE MUTEX EVENT MEM_DFPXM UI DATE PERF TIMEOUT LAPBIND DKI
MEM: memory device 'sys_bank' vaddr 0x7bc17000 size 0x1a4000
[
.......... more lines ]..........
C_INIT: started
C_INIT: /image/sys_bank mounted on /dev/bd00
C_INIT: found /image/sys_bank/sysadm.ini
C_INIT: executing start-up file /image/sys_bank/sysadm.ini
[
.......... more lines ]..........
C_INIT: rshd started
2.5. Booting the ChorusOS system 39
For the carbon system (869DA806 is used instead of 869DA806.ChorusOS.4.0),
we have:
RAM size: 0x1000000 bytes

ChorusOS r4.0.0 for Intel x86 - Intel x86 PC/AT

My IP 134.157.168.6, RARP Server IP 134.157.168.9

Loading file 869DA806.ChorusOS.4.0 on server 134.157.168.9: failed!

Loading file 869DA806 on server 134.157.168.9: loaded!

Loading file chorus_vm on server 134.157.168.9: loaded!


[
.......... more lines ]..........
C_INIT: rshd started

Before creating the C INIT actor, the system


- mounts a pseudo root le system;
- creates a /dev special les directory in which it creates a console node;
- creates a /image directory into which it mounts the contents of the system
image as a read-only, FAT 12 le system in /image/sys bank.
These operations provide the C INIT actor with access to the console and to
the contents of the system image.
Chapter 3
Getting started
3.1 The C INIT actor and its initialization
3.1.1 Overview
Once the system image has been loaded, all embedded actors are created and
started on the target system; among them the C INIT actor, if the extended
environment has been used. This actor is intended to allow users to communi-
cate with the target by using the rsh command; the user can, for example, get
information about the system, create and load or kill actors dynamically.
As the C INIT actor is intended to access remote les, it is important to note
that the two cinit.defaultUid and cinit.defaultGid tunable parameters
(whose default value is 0) de ne the credentials of the C INIT actor. Thus,
with the default value of the credentials, a le can only be remotely accessed by
C INIT if it has the permissions for everybody for the corresponding request.
Di erent types of initializations can be performed by the C INIT actor when
it is started. These are done by reading the contents of les that have been
included in the archive le or are accessed remotely on the host, and by using
values of environment variables de ned in the archive.
These initializations correspond to the execution of built-in commands which
are presented in the next sections. The execution of these commands may
require the services of other system actors (ADMIN, AM or IOM).
3.1.2 The sysadm.ini initialization le
The /image/sys bank/sysadm.ini is used by C INIT at boot time to initialize
le and network devices. It may include di erent commands like mkdev, route
or mount. It is important to note that this le is not read remotely but is
41
42 Chapter 3. Getting started
included in the system image archive. This le is an embedded version of the
BUILD_DIR/conf/sysadm.ini le.

3.1.3 Mounting the root le system


In section 2.3.4 we built a root le system for the target system and described
its contents. The root directory in the build dir directory is the root of this le
system and is intended to be NFS mounted as root (/) of the le system of the
target system using the mount built-in command. In release 4 of the system,
the mount operation invokes the ADMIN actor through a lap.
First of all, the NFS CLIENT feature must have been been con gured on the
target system. From the host point of view the build dir/root directory must
have been exported and the nfsd daemon must be active. Therefore, in the
/etc/dfs/dfstab le we add the line
share -F nfs -d "chorus root" -o rw /home/jmr/CHORUS/r4/build/root

We then start the nfs server by using the nfs.server script in the /etc/init.d
directory with the start argument. It creates the /etc/dfs/sharetab le of
exported les by executing the share commands in the /etc/dfs/dfstab le:
--> su
Password: (= provide root's password
# /etc/init.d/nfs.server
# exit
--> cat /etc/dfs/sharetab
/home/jmr/CHORUS/r4/build/root - nfs rw
-->
If the nfsd daemon is already active, we simply execute the share command. If
no root le system is mounted, C INIT displays the following warning message
C_INIT: warning - root file system not mounted

In this case, the root le system must be mounted explicitly using the mount
built-in command. In our example, the root of the le system of the target
named neon is the /home/jmr/CHORUS/r4/build/root directory of the host
system whose name is fluorine and whose IP address is 134.157.168.9:
--> rsh neon mount 134.157.168.9:/home/jmr/CHORUS/r4/build/root /
134.157.168.9:/home/jmr/CHORUS/r4/build/root on / (nfs)
-->

It is possible to perform this operation automatically when booting the system


by adding the line
mount 134.157.168.9:/home/jmr/CHORUS/r4/build/root /

at the end of the $BUILD_DIR/conf/sysadm.ini before building the archive.


3.1. The C INIT actor and its initialization 43
3.1.4 About security: the /etc/security le
The C INIT rsh daemon authenticates users issuing commands from the host
through the rsh command. This authentication procedure is based on the exis-
tence and the contents of the /etc/security le in the root le system of the
target.
Once the root le system has been mounted, a di erent message will be dis-
played by C INIT depending on whether the /etc/security le exists and is
readable by C INIT or not.
 If the le exists and is readable, the message is:
C_INIT: system in secured mode

In this case, the secure mode of the system is activated.


The lines of this le have the following format:
username:trusted:uid:gid:group list:host list
where
- username is the user's name on a remote system. The asterisk ("*")
character is interpreted as "all users";
- trusted is either the TRUSTED string if the entry corresponds to a trusted
user, otherwise an empty string;
- uid is the user identi cation number;
- gid is the group identi cation number;
- group list is a list of additional group identi ers separated by commas;
- host list is a list of IP-addresses of remote systems separated by commas.
The asterisk ("*") character is interpreted as "all hosts".
We will use the following contents of the /etc/security le in the examples
when running the systems in secured mode:
jmr:TRUSTED:1001:101::134.157.168.9
rifflet::1002:101::134.157.168.9
*:::::*
Given these contents of the /etc/security le:
- the user whose identi cation is jmr is recognized as a trusted user (uid is
1001 and gid is 101) on the system whose IP-address is 134.157.168.9;
- the rifflet user on the same system (IP-address is 134.157.168.9) is
recognized as a non-trusted user (uid=100 and gid=101);
- all other users are recognized as non trusted users from any host.
44 Chapter 3. Getting started
In this case, an authentication procedure is activated which fails:
- when a request is issued by a user whose name does not appear in the
/etc/security le;
- if the remote host from which the user issued the command does not ap-
pear in the list of recognized hosts for that user. In that case, a speci c
message is sent back and the command is aborted.
If the authentication procedure succeeds, the user's privileges (uid and gid) are
extracted from the le. If the user is trusted, he inherits the credentials of
C INIT which give access to the full set of C INIT commands.
 If the le does not exist or is not readable, the message is:
C_INIT: notice - system not in secured mode
The ChorusOS system will be running in non secured mode: every user inherits
the credentials of C INIT and is considered a trusted user (he may create any
type of actor and has access to all the commands). All the actors that will be
created by the AM actor (called c actors or extended actors) will be owned by
the user whose uid is cinit.defaultUid.
3.1.5 The startup les
Finally, if the /etc/rc.chorus.IPAddr le exists (/etc/rc.chorus.869DA80A)
in the root le system that was mounted, it is executed in the context of the
C INIT actor, that is, as if it was executed through the source built-in com-
mand. If that le is not available, the /etc/rc.chorus le, if any, is executed.
This startup le would typically contain speci c mount operations (see 3.6) such
as:
mount 134.157.168.9:/home/jmr/CHORUS/bin86 /mnt

3.2 List of C INIT built-in commands


We rst give the list of the built-in commands of C INIT and will present
the most useful ones. All these commands are directly recognized by the rshd
interpreter. The list of these commands is accessed by using the help command:
--> rsh neon help
C_INIT ChorusOS 4.0.0- valid commands that deal with:
File Systems:
mount [[-t nfs|ufs|msdosfs|pdevfs] host:pathname|special_file
[mount_point]]
umount [-v|-F|-f|-a|-t nfs|ufs|msdosfs|pdevfs] [special_file]
swapon [mount_point]
3.3. The echo command 45
Actors:
arun [-g rgid] [-S | -U] [-k] [-T] [-d] [-q] [-D] [-Z] [-xip] path
[args]
akill [-s site] {-g rgid | [-c] aid }
aps
umask [mode]
ulimit [-HSafn] [limit]
Environment variables:
setenv var value
unsetenv var
env
Networks:
route
netstat
ping host
ifconfig
ifwait ifname [timeout, default infinite]
rarp ethernet_interface_name
pppd
pppclose device
pppstop
ethIpcStackAttach [dtreepath]
Devices:
mknod name [b | c] major minor
dtree
mkdev name unit [dtreepath]
This Target:
reboot
restart
memstat
chorusStat
shutdown -i 0|1|2|3
This shell:
echo string
source filename
sleep [time in seconds, default=1s]
help
console
rshd
-->

3.3 The echo command


This command simply echoes its arguments to the standard output. It may be
used, for example, to display messages in initialization les while they are being
executed.
46 Chapter 3. Getting started
3.4 The environment: env, setenv and unsetenv
The environment is de ned as a set of pairs (name, value). The list of these
pairs is listed by calling the env command:
--> rsh neon env
PATH=/bin
HOST=
TZ=GMT0
-->
A new variable may be de ned or the value of an existing one may be changed
by using the setenv command:
--> rsh neon setenv PERSO Hello_from_ChorusOS
--> rsh neon setenv PATH /bin:/mnt
--> rsh neon env
PATH=/bin:/mnt
HOST=
TZ=GMT0
PERSO=Hello_from_ChorusOS
-->
Finally, a variable may be removed from the environment by using the unsetenv
command:
--> rsh neon unsetenv PERSO
--> rsh neon env | grep PERSO
--> (= PERSO is no longer de ned in the environment

3.5 Statistics: memstat and chorusStat


3.5.1 The memstat command
This command gives information on current memory usage on the target system.
The number of bytes for the total memory size, the current free memory and
the current locked memory are displayed.
--> rsh neon memstat
memory size 16384000 free 13377536 locked 3006464
-->

3.5.2 The chorusStat command


This command is available as a C INIT built-in command if the ADMIN CHORUS-
STAT has been selected, and as a standalone command on every system (binary
located in the /bin directory):
3.6. The mount and umount commands 47
--> rsh neon chorusStat (=
ADMIN CHORUSSTAT selected for neon
started aid = 2
site physical memory: 0xfa1000 (16388096)
memory version: MEM_VM
allocated physical memory: 0x38c000 (3719168)
unallocated physical memory: 0xc15000 (12668928)
lid priv #threads text data dyn name
1 SUP 8 1744896 380928 417792 kern
3 SUP 0 12288 4096 0 D_pci
4 SUP 0 8192 4096 0 D_pcienum
[ ]
........ one line for each actor .....
17 SUP 0 8192 4096 0 PD
18 SUP 2 438272 24576 327680 N_iom
19 SUP 0 81920 4096 40960 AM
20 SUP 0 114688 8192 32768 ADMIN
21 SUP 1 94208 8192 8192 C_INIT
--> rsh gallium chorusStat (=
ADMIN CHORUSSTAT not selected for gallium
C_INIT: chorusStat: service unavailable
--> rsh gallium arun chorusStat
started aid = 2
site physical memory: 0xba1000 (12193792)
memory version: MEM_VM
allocated physical memory: 0x37e000 (3661824)
unallocated physical memory: 0x823000 (8531968)
.......................................
-->

3.6 The mount and umount commands


We already mentioned and used the mount command to mount the root le
system of the target system.
We assume now that the CHORUS variable of our shell environment corresponds
to the root of the les' subtree where we develop new ChorusOS applications,
(in the example given below, its value is /home/jmr/CHORUS) and that we use
the bin86 subdirectory to install the binary les we intend to execute on x86
targets (neon, for example).
We assume that we created a /mnt directory on the target system and that we
exported the $CHORUS/bin86 directory to the host system (as we did for the
root directory).
We use this /mnt directory to mount the /home/jmr/CHORUS/bin86 directory:
--> rsh neon mount 134.157.168.9:$CHORUS/bin86 /mnt
134.157.168.9:/home/jmr/CHORUS/bin86 on /mnt (nfs)
-->
On a secured system, a le system can only be mounted by trusted users.
48 Chapter 3. Getting started
The mount command can also be used without any argument to display the list
of the le systems that have been mounted:
--> rsh neon mount
root_device on / (pdevfs)
devfs on /dev (pdevfs)
devfs on /image (pdevfs)
/dev/bd00 on /image/sys_bank (msdos)
134.157.168.9:/home/jmr/CHORUS/r4/build/root on / (nfs)
134.157.168.9:/home/jmr/CHORUS/bin86 on /mnt (nfs)
-->

We can observe that, apart from the two remote le-systems we mounted, the
neon target system has di erent local or RAM le-systems.
The umount command is used by a trusted user to unmount a le-system:
--> rsh neon umount /mnt
--> rsh neon mount
root_device on / (pdevfs)
devfs on /dev (pdevfs)
devfs on /image (pdevfs)
/dev/bd00 on /image/sys_bank (msdos)
134.157.168.9:/home/jmr/CHORUS/r4/build/root on / (nfs)
-->

3.7 The aps command


This displays the list of all the c actors (that is, those actors created or known
of by the actor manager). For each actor the following information is given:
 UID: the uid of the user on behalf of which the c actor is running;
 AID: the actor's local identi er;
 NAME: the name of the actor (it is the value of argv[0] when calling the main
function);
 DBG: 1 if the actor is in debug mode and 0 otherwise;
 GROUP: information related to the hot-restart feature.
We execute the command on the neon target system just after the ChorusOS
system has been booted:
--> rsh neon aps
UID AID NAME DBG GROUP
0 21 C_INIT 0 N/A
0 20 ADMIN 0 N/A
-->

These results show that there are two c actors on the booted system.
3.8. The arun command 49
3.8 The arun command
3.8.1 General form of the arun command
This command, whose call has the general form
arun [-g rgid] [-Sj-U] [-k] [-T] [-d] [-q] [-D] [-Z] actor name [actor args]
is used to create new c actors.
In the command:
 actor name is the pathname of a binary executable le which should be loaded
and executed by the newly created actor. If the pathname is not absolute, the
le is searched according to the value of the PATH environment variable. As
seen before, the default value of this variable is /bin : it allows the execution
of standard Unix-like commands located in that directory (cp, ls, mkdir, . . . )
or speci c ChorusOS commands (chorusStat or cs);
 actor args is an optional list of arguments that will be transmitted as an argv
argument when the main thread of the actor calls the main function.
3.8.2 What the command does
The e ect of the arun command is to create an actor. The C INIT actor relies
on the services of the actor manager, which in turn invokes the appropriate
services of the IOM and of the kernel. As a consequence, an actor is created by
the AM actor and, is called a c actor (or extended actor). This type of actor
is a CHORUS actor which owns resources and has some additional attributes:
 it has a local identi er on the site (aid) which is displayed on the screen. If
none of the -S, -U or -T options are used, the privilege of the actor is automat-
ically deduced from the le which is loaded; it may be a user actor ( le with
the normal extension u, produced by linking the user library) or a supervisor
actor ( le with the normal extension s.r, produced by linking the supervisor
library). The use of one (and only one) of the options de nes the type of the
actor (-U corresponds to a user actor, -T to a trusted user actor [system actor]
and -S to a supervisor actor). If one of the options is used, it must be compat-
ible with the type of code corresponding to the loaded le. In any case, only a
trusted user can create a system or a supervisor c actor;
 it has a memory context which is created for that actor, the code is loaded
and a data region and a stack region are created;
 a thread executing a call to the main function of the code is created (it will
be called the main thread of the c actor). The call to the main function is
done using the list of arguments actor arguments. This thread has a default
scheduling class and a default priority which were de ned when the system was
built.
50 Chapter 3. Getting started
On the neon target, the default con guration has been selected (the main thread
belongs to the FIFO scheduling class and its priority is 140);
 it has a le context: it has a standard input, a standard output, a standard
error output, a root and a current directory. This property is speci c to actors
created using the arun command or created by any actor through the afexec
system calls we will present in chapter 6.
The other options of the command have the following e ects:
-k: if the new c actor is a supervisor actor, its symbol table will be
accessible by the kernel debugger (see section 4.3) otherwise, the option
is ignored;
-d: the new c actor is created in the stopped state (used by some debug-
gers).
The execution is synchronous: on the host machine, the remote shell waits for
the termination of the actor. However, it is possible to spawn other actors even
though the rst one has not yet terminated.
3.8.3 Executing standard commands: the example of cs
We rst illustrate the creation and loading of a new actor and the creation and
activation of a main thread by using the arun command with the cs ChorusOS
utility. This command is located in the /bin directory: it gives internal infor-
mation (from the kernel's point of view) for all the actors currently existing in
the system. This command can only be executed by trusted users (since it gives
information like the capabilities of actors).
3.8.3.1 The cs command without argument
Firstly, the command may be called without any argument as in the following
example:
--> rsh neon arun cs
started aid = 2
ChorusOS r4.0.0 Site 0 Time 1h 38m 46
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000001 869da80a 00000001 00000001 0001 SUP STARTED 003 kern
2000001b 869da80a 00000002 00000001 0002 SUP STARTED 001 cs
20000003 869da80a 00000003 00000000 0003 SUP STARTED 000 D_pci
20000004 869da80a 00000004 00000004 0004 SUP STARTED 000 D_pcienum
20000005 869da80a 00000005 00000001 0005 SUP STARTED 000 D_pci_pic
20000006 869da80a 00000006 00000007 0006 SUP STARTED 000 D_pci_isa
20000007 869da80a 00000007 00000000 0007 SUP STARTED 000 D_i8254
20000008 869da80a 00000008 00000008 0008 SUP STARTED 000 D_mc146818
20000009 869da80a 00000009 00000001 0009 SUP STARTED 000 D_ns16550
3.8. The arun command 51
2000000a 869da80a 0000000a 0000000b 0010 SUP STARTED 000 D_epic100
2000000b 869da80a 0000000b 00000000 0011 SUP STARTED 000 D_dec21x4x
2000000c 869da80a 0000000c 0000000c 0012 SUP STARTED 000 D_el3
2000000d 869da80a 0000000d 00000001 0013 SUP STARTED 000 D_smc1660
2000000e 869da80a 0000000e 0000000f 0014 SUP STARTED 000 D_ne2000
2000000f 869da80a 0000000f 00000000 0015 SUP STARTED 000 D_sram
20000010 869da80a 00000010 00000010 0016 SUP STARTED 000 D_idebios
20000011 869da80a 00000011 00000001 0017 SUP STARTED 000 PD
20000012 869da80a 00000012 00000013 0018 SUP STARTED 002 N_iom
20000013 869da80a 00000013 00000000 0019 SUP STARTED 000 AM
20000014 869da80a 00000014 00000014 0020 SUP STARTED 000 ADMIN
20000015 869da80a 00000015 00000001 0021 SUP STARTED 001 C_INIT
-->
After listing general information about the system (version, site's identi cation
and time information), it gives the following information for every existing actor
on the site (all actors, and not only c actors):
 ACTOR-UI: actor's unique identi er expressed as two 4-byte numbers (head
and tail) in hexadecimal format;
 KEY: key of the actor's capability expressed in the same form as the unique
identi er. The UI and the key form the actor's capability;
 LID: actor's local identi er: each actor is locally identi ed on the site it belongs
to by a small integer;
 TYPE: actor's type: SUP for a supervisor actor and USER otherwise (for a user
actor or a system actor);
 STATUS: actor's status (STARTED or STOPPED);
 TH#: number of threads belonging to the actor;
 NAME: actor's symbolic name, if any.
In our example, we can see the di erent system actors we presented in chapter
1 and the actor created for executing the cs command itself; we can also see
that this actor is a supervisor actor.

3.8.3.2 Getting information on system con guration


The following options are available:
 -lM: the various modules of the kernel are printed (high level features are not
listed):
--> rsh neon arun cs -lM
started aid = 2
ChorusOS r4.0.0 Site 0 Time 1h 45m 15
CORE SCHED_FIFO SEM MIPC IPC_L MEM_PRM KDB TICK MON ENV ETIMER LOG LAPSAFE
MUTEX EVENT UI DATE PERF TIMEOUT LAPBIND DKI
52 Chapter 3. Getting started
 -lE: the environment of the kernel is printed:
--> rsh neon arun cs -lE
started aid = 2
ChorusOS r4.0.0 Site 0 Time 1h 46m 59
OS_CONF=:IOM:AM:ADMIN:C_INIT
-->

3.8.3.3 Getting the system's statistics


When called with the -lr option, the command prints statistics on kernel re-
sources.
--> rsh neon arun cs -lr
started aid = 2
ChorusOS r4.0.0 Site 0 Time 2h 19m 9
NAME TYPE MODE I_SIZE CUR HIGH MEMCUR MEMHIGH
LAPSAFE POOL DYNAMIC 0x0030 0010 0010 0x00001000 0x00001000
ETIMER_LI POOL STATIC 0x0004 0000 0000 0x00000000 0x00000000
ETIMER POOL DYNAMIC 0x0068 0000 0000 0x00000000 0x00000000
UTimerQueue POOL DYNAMIC 0x0000 0000 0000 0x00000000 0x00000000
RGN POOL DYNAMIC 0x0018 0059 0061 0x00001000 0x00001000
CTX POOL DYNAMIC 0x0024 0002 0002 0x00001000 0x00001000
LAPBIND POOL DYNAMIC 0x0020 0007 0007 0x00001000 0x00001000
ACTOR POOL DYNAMIC 0x013c 0022 0022 0x00002000 0x00002000
THREAD POOL DYNAMIC 0x018c 0007 0009 0x00001000 0x00001000
IPC_MEMORY MEM DYNAMIC -- -- -- 0x00000000 0x00000000
SYSSTACK MEM DYNAMIC -- -- -- 0x00015000 0x0001b000
KNINITMEM MEM STATIC -- -- -- 0x000fe000 0x000fe000
MIPCMSG MEM DYNAMIC -- -- -- 0x00000000 0x00000000
UserQueue POOL DYNAMIC 0x0020 0006 0008 0x00001000 0x00001000
HOST => 0xfa0000, FREE => 0xca9000, KERN => 0x0
HOST => 0xfa0000, FREE => 0xca9000, KERN => 0x0
-->

3.8.3.4 Getting unique identi ers


The -lu option gives the list of all the local unique identi ers which are currently
known on the local site:
--> rsh neon arun cs -lu
started aid = 2
ChorusOS r4.0.0 Site 0 Time 1h 49m 8
LOCAL UI TABLE:
HASH-LIST UI
010 00000000 869da80a
[ ]
.... more lines ....
010 2000001f 869da80a
-->
3.8. The arun command 53
3.8.3.5 Getting information on scheduling
When called with the -ls option, the command displays information on the
current status and run queues of the scheduler:
--> rsh neon arun cs -ls
started aid = 2
ChorusOS r4.0.0 Site 0 Time 1h 54m 25

CURRENT THREAD: 9 PRIORITY: 140


RUN QUEUE:
HIGH WATERMARK: 256
PRIORITY THREAD-LI ACTOR-UI KEY
-->

3.8.3.6 Getting information about one thread


When called with the option -lt threadLi, the following information is given
for the thread whose local identi er is threadLi:
 THREAD-LI: thread's local decimal identi er;
 PRIORITY: thread's priority (a value in the range [0:255]);
 TT: total CPU time consumed by the thread expressed as a decimal number
of milliseconds;
 IT: total CPU time consumed by the thread in the actor where it was created,
expressed as a decimal number of milliseconds (see chapters 11 and 15);
 CTX: hexadecimal address of an internal structure associated with the thread;
 SC-MS-PN: internal information for the thread expressed as three hexadecimal
numbers:
- SC: the thread suspend counter;
- MS: the thread's mask set;
- PN: the thread's pending ags set;
 NAME: thread's symbolic name, if any.
3.8.3.7 Getting information about IPC ports
The command can be called with a number of options to obtain information
about ports. When called:
 with the -lpa option the command displays information for all the ports
located on the local site;
 with the -lpm option the command displays information for all the ports
located on the local site and whose message queue is not empty;
 with the -lp portLi option, the command displays information for the port
whose local identi er is portLi.
54 Chapter 3. Getting started
The following information is displayed for the ports:
 PORT-UI: port unique identi er;
 PORT-LI: port local identi er;
 ENABLED: port state (yes or no);
 CTX: port context address (hexadecimal);
 MSG#: number of messages in port queue;
 ACTOR-UI: unique identi er of the owning actor.
3.8.3.8 Getting information about one actor
When called the -la actorLi option, the command prints the list of kernel
resources of the actor whose local identi er is actorLi. The information is given
for all threads and all ports in the actor as described above, and information is
given for all actor's memory regions:
 START: region's starting virtual address;
 SIZE: region's size;
 OFFSET: o set (expressed as a hexadecimal number of bytes) of the region in
the corresponding segment;
 ALLOC: amount of physical memory (expressed as a hexadecimal number of
bytes) allocated to the region;
 OPTIONS: region's attributes (WR EX SU NS FZ ND NW AW ST IS IC ).
In the next sequence, we display the di erent attributes of the C INIT actor:
--> rsh neon arun cs -la 21
started aid = 2
ChorusOS r4.0.0 Site 0 Time 1h 56m 38
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000015 869da80a 00000015 00000001 0021 SUP STARTED 001 C_INIT
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0004 030 00000000 00000000 ff6960 0- 1- 0 rshd_handler
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
20000015 869da80a def no fac460 0 20000015 869da80a
START SIZE OFFSET ALLOC OPTIONS
400e4000 00017000 00000000 00017000 EX SU
400fb000 00002000 00000000 00002000 WR SU
7bbfd000 00002000 00000000 00002000 WR SU
7bc03000 00002000 00000000 00002000 WR SU
-->

3.8.4 Dynamic loading of user applications


We are now interested in dynamically loading and executing binary les that we
produced on the host system (this will be explained in chapter 4). As previously
mentioned, we assume that the binary les are installed on the host system in
3.8. The arun command 55
a directory $CHORUS/bin86 that was NFS mounted on the /mnt directory on
the target systems, as described earlier.
We now assume that we produced and installed the binary les corresponding
to the following applications on the host system in the $CHORUS/bin86 direc-
tory:
 The trivialAct application simply prints a message and the list of its pa-
rameters (argv parameter of the main function) and terminates. It corresponds
to the following C program:
--> cat trivialAct.c
#include <stdio.h>
main(int argc, char *argv[ ]) {
int ind;
printf("Hello! I am the trivial actor\n");
if(argc == 1)
return;
printf("I have %d parameters\n", argc - 1);
for(ind = 1; ind < argc; ind ++)
printf(" ==> %s\n", argv[ind]);
}
-->
Two versions have been installed:
- trivialAct u: will be executed as a user actor;
- trivialAct s.r: will be executed as a supervisor actor.
 Within the noDelay1 application, the main thread of the actor simply enters
the sleeping state for an in nite period. It does this by calling the related service
of the executive which corresponds to a call of the threadDelay primitive of
the executive's API:
--> cat noDelay1.c
#include <chorus.h>
main( ) {
threadDelay(K_NOTIMEOUT); /* the calling thread sleeps for ever */
}
-->

This program is equivalent to the C program


main( ) {
sleep(-1);
}

for execution under a Unix system.


Two versions (noDelay1 u and noDelay1 s.r) have also been installed.
56 Chapter 3. Getting started
 First, we try to execute the user version of the trivialAct application with
the default value of the PATH variable on the target system:
--> rsh neon mount 134.157.168.9:/home/jmr/CHORUS/bin86 /mnt
134.157.168.9:/home/jmr/CHORUS/bin86 on /mnt (nfs)
--> rsh neon env | grep PATH
PATH=/bin
--> rsh neon arun trivialAct_u aaa bbb
C_INIT: afexec of trivialAct_u failed - No such file or directory
-->

The trivialAct u le cannot be accessed as it does not belong to a directory in


the list de ned by the PATH variable of the target environment. Two solutions
are possible: either giving its absolute pathname (/mnt/trivialAct_u ) or
modifying the PATH variable (by adding the mnt directory):
--> rsh neon arun /mnt/trivialAct_u aaa bbb (= give absolute path
started aid = 2
Hello! I am the trivial actor
I have 2 parameters
==> aaa
==> bbb
--> rsh neon setenv PATH /bin:/mnt (= add /mnt in PATH
--> rsh neon arun trivialAct_u aaa bbb ccc (= use relative pathname
started aid = 2
Hello! I am the trivial actor
I have 3 parameters
==> aaa
==> bbb
==> ccc
-->

 We now try to execute the supervisor version of the trivialAct application


on a secured system (where jmr is a trusted user and rifflet is not):
(rifflet)--> rsh neon arun trivialAct_s.r
C_INIT:afexec of trivialAct_s.r failed -Not owner
(rifflet)-->

(jmr)--> rsh neon arun trivialAct_s.r


started aid = 2
Hello! I am the trivial actor
(jmr)-->

Note that the type of the actor (user or supervisor) to be created is deduced
from the contents of the loaded les, there is no need to use the -S option to
create a supervisor actor.
3.8. The arun command 57
 The request to create a new actor with arun may also be launched in back-
ground:
--> rsh neon arun trivialAct_u aaaaa bbbbb &
[1] 5713 (=
message from local shell
-->
We can observe that nothing happens; in fact the UNIX process on the host
system is stopped due to a tty input (message [1] Stopped (SIGTTIN) ...)
(even if no reads are posted by the remote command). This type of stopped
process has to be restarted in foreground mode:
--> fg %1
rsh neon arun trivialAct_u aaaaa bbbbb
started aid = 2
Hello! I am the trivial actor
I have 2 parameters
==> aaaaa
==> bbbbb
-->
A better solution is to use the -n option of the rsh command; the standard
input is then redirected to /dev/null and the command executes normally:
--> rsh -n neon arun trivialAct_u xxxxx yyyyy &
[1] 5716 (=
message from local shell
--> started aid = 2 (=
shell's prompt is displayed
Hello! I am the trivial actor
I have 2 parameters
==> xxxxx
==> yyyyy
It is important to note that if an actor is created using the rsh command with
the -n option, threads in that actor cannot read data on the tty.
 We have now created, on the host system, two shell aliases (here csh or
tcsh) named neon and neon-n (as well as others deduced from the names of
other target systems like carbon) for the commands rsh neon arun and rsh
-n neon arun. These will be used to create and load actors on the target
machine with les extracted from the $CHORUS/bin86 directory (here the value
of the CHORUS variable is /home/jmr/CHORUS):
--> alias
..........
neon rsh neon arun /home/jmr/CHORUS/bin86/!*
neon-n rsh -n neon arun /home/jmr/CHORUS/bin86/!*
carbon rsh carbon arun /home/jmr/CHORUS/bin86/!*
carbon-n rsh -n carbon arun /home/jmr/CHORUS/bin86/!*
..........
58 Chapter 3. Getting started
--> neon trivialAct_u A B
started aid = 21
Hello! I am the trivial actor
I have 2 parameters
==> A
==> B
-->
 We now use the noDelay1 application which, unlike the previous trivialAct
one, does not terminate; it allows us to use standard utilities we presented
before to examine the state of the system. We assume here that we are using a
non-secured system:
--> neon-n noDelay1_u &
[1] 5752 (= message from local shell
--> started aid = 2 (= local id of the new actor
The new actor appears in the results of the aps command:
--> rsh neon aps
UID AID NAME DBG GROUP
0 2 noDelay1_u 0 N/A
0 21 C_INIT 0 N/A
0 20 ADMIN 0 N/A
-->
Complete information concerning the new actor is retrieved by using the cs
standard utility:
--> rsh neon arun cs -la 2
started aid = 22 (=
local id of the actor executing cs
ChorusOS r4.0.0 Site 0 Time 3h 20m 51
ACTOR-UI KEY LID TYPE STATUS TH# NAME
2000002e 869da80a 00000002 00000000 0002 USER STARTED 001 noDelay1_u
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0009 140 00000000 00000000 ff61a4 0- 1- 0 main
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
2000002e 869da80a def no fac324 0 2000002e 869da80a
START SIZE OFFSET ALLOC OPTIONS
ffff4000 00006000 00000000 00006000 WR
ffffb000 00001000 00000000 00001000 WR
ffffc000 00003000 00000000 00003000 EX
-->

- we get the capability of the actor expressed as a hexadecimal value and its
local identi cation: here 2000002e 869da80a 00000002 00000000 and 2. The
actor is a user actor and is started;
- the actor has one thread (the main thread) whose local identi er is 9;
- the actor has a default port where there is no pending message;
3.9. The akill command 59
- the actor has three memory regions (data, stack and text). It is worth noting
that the system is using the MEM PROTECTED memory management model
(the VIRTUAL ADDRESS SPACE feature is active); di erent results would be ob-
tained with the MEM VIRTUAL model (see chapters 7 and 15).
We also create a supervisor actor for the application and extract the corre-
sponding line produced by the cs command:
--> neon-n noDelay1_s.r &
[2] 5758
--> started aid = 22
--> rsh neon arun cs | grep noDelay1_s.r
started aid = 23
20000030 869da80a 00000016 00000084 0022 SUP STARTED 001 noDelay1_s.r
-->

3.9 The akill command


This command allows c actors to be killed. On a secured system a non trusted
user can only kill actors running with the same uid. Here, we use the command
as a trusted user to kill the two actors that we created previously:
--> rsh neon akill 2 (= kill the user actor
--> rsh neon akill 22 (= kill the supervisor actor
--> rsh neon aps
0 21 C_INIT 0 N/A
0 20 ADMIN 0 N/A
-->

3.10 The ping command


Called as ping IPaddr, it sends an ICMP datagram to the system with the IP
address speci ed requesting an answer.

3.11 The reboot command


All the c actors on the target system are killed (as if akill had been called for
every c actor). All le systems are unmounted, the C INIT actor itself is killed
and the system is nally rebooted. If the system is running in secured mode,
this command is restricted to trusted users.
--> rsh reboot
shutdown in progress ...
-->
After a few seconds, the system is restarted:
60 Chapter 3. Getting started
--> rsh neon arun cs -lM (=
wait until system restarts
started aid = 2
ChorusOS r4.0.0 Site 0 Time 4
CORE SCHED_FIFO SEM MIPC IPC_L MEM_PRM KDB TICK MON ENV ETIMER LOG LAPSAFE
MUTEX EVENT UI DATE PERF TIMEOUT LAPBIND DKI
-->

3.12 The kernel debugger: kdb


3.12.1 Overview
The kernel includes a debugging mode that allows debugging of the kernel itself
as well as supervisor actors for which a system table is available. An actor
created dynamically using the arun utility may also be debugged with the kernel
debugger if the -k option of the utility was used when creating it.

3.12.1.1 Entering the kernel debugger


When it enters the kernel debugger, the kernel rst saves the register context
and displays a prompt.
The kernel debugger may be entered
 when a thread of a supervisor actor or of the kernel itself reaches a breakpoint
previously set from kdb;
 by sending the character <CTRL> A on the serial line connected to the PC/AT
target. The register context contains the values of the registers when the signal
is received;
 when a panic situation occurs from which the kernel cannot recover.
3.12.1.2 Exiting the kernel debugger
The c command of kdb resumes execution. The register context saved when
entering the debugger is used for restoring the registers.

3.12.1.3 The help command


The help command is accessed by h (or H or ?) and allows information for a
speci c command to be retrieved.

3.12.2 List of kdb commands


The following table gives a brief list of the commands available for kdb:
3.12. The kernel debugger: kdb 61
command interpretation
1 one step
f j j j j g [exp ]
b cdrsw software breakpoints management
bc: sets a breakpoint
bd: deletes a breakpoint
br: lists all breakpoints
bs: disables a breakpoint
bw: enables a breakpoint
Bn clear breakpoint number n
c resumes execution
reg fnewValueg display or modify register
for ix86, reg may be eax, ebx, ecx, edx
esi, edi, ebp, eip, efl
f [address] nds nearest symbol close to address
l[ajmjsjt list kernel entities
similar to cs utility
n executes next call as a step
on set line number for more pages to n
pn stop at next call or n steps
R dump current registers
rfbjwjlg [readExp [itemNbr]] read memory
readExp is the start address,
itemNbr is the number of items
b is for byte, l for two bytes, w for word
reboot reboot local site
sjSjstjST [threadLi] display thread's stack
stajSTA display stacks of all threads
t [count] step for several count instructions
ut displays time from boot
wfbjwjlg exp value [itemNbr] write in memory
exp de nes address of rst address
value is a hexadecimal value
= shows and disassembles current PC
Chapter 4
Producing ChorusOS
applications
4.1 Overview
A ChorusOS system provides an environment for executing applications. We
have seen in previous chapters how it is possible to run applications either
by embedding them with the system in a bootable archive or by dynamically
loading them by using the remote shell protocol and the arun command. This
approach is commonly used to develop, test and debug applications before even-
tually embedding them.
This chapter describes how to write new components to be run on ChorusOS:
it may be a user application, a device driver or a BSP. We assume that we will
be developing applications on the Solaris host system where we installed the
SEW as described in chapter 2, and that the target system is an ix86 based
computer. The production of binary applications is achieved by using di erent
tools accessed through the install dir/4.0/chorus-x86/tools/host/bin direc-
tory. This directory should therefore belong to the list of directories de ned by
the PATH variable of the shell environment on the host system.
The production of a binary corresponding to a set of source les requires several
steps:
 writing an Imakefile. This type of le is a high-level Makefile which con-
tains C-processor macros. The imake command builds a Makefile from an
Imakefile by calling the C preprocessor. An imake environment is de ned by
various les located in the install dir/4.0/chorus-x86/tools/imake directory
(as well as les speci c to the application); this environment de nes di erent
variables and production rules used to write Imake les;
63
64 Chapter 4. Producing ChorusOS applications
 a Makefile is built by the ChorusOSMkMf command which interprets the
rules of the imake environment and calls the standard imake command. A
make environment is used which it corresponds to the set of tools in the cross
development system (including a compiler, a link editor, . . . ): the gcc compiling
tools are usually selected and the install dir/4.0/chorus-x86/tools/tgt-make
directory contains di erent les de ning the variables and rules used to build
the component. This environment de nes the rules used to build a binary. It
de nes variables which correspond to the location of include les or libraries;
 the component is then built using the standard make command.
On-line manuals are available for the programming interface and the cross-
development system tools. To get access to these manual pages using the man
Unix command, the directory install dir/4.0/chorus-doc/man should be added
to the list of directories de ned by the MANPATH variable of the user's environ-
ment.
Man pages may also be accessed through the html subdirectory of the directory
chorus-doc with a web browser.

4.2 Organization of the working space


This book is illustrated by examples developed on a Solaris workstation. We
organized our working space as a tree whose root's absolute pathname matches
the value of the CHORUS variable in the shell environment; in this book it is the
/home/jmr/CHORUS directory.
This directory contains a number of subdirectories:
 sources: is the root of a subtree containing the source les;
 bin86: contains the binaries executable on an ix86 target in either user or
supervisor mode. The usual convention for naming a le for execution in a
given mode is to use u as a le sux for user actors and s.r for supervisor
actors: we use this convention in the book. We have seen that this directory
was exported and mounted on the target system on /mnt;
 lib86: contains the libraries we develop for ix86 based target systems;
 include: contains header les for libraries.
4.3 General conventions for writing C or C++
codes for ChorusOS
When writing applications in C or C++, the programmer is using di erent
prede ned components such as header les, where symbolic constants and types
are de ned, and which allow a correct use of the interface with the modules of
4.3. General conventions for writing C or C++ codes for ChorusOS 65
the kernel. The application invokes services provided by the kernel or speci c
boot actors by calling functions belonging to standard libraries. We describe
here the main conventions used for naming these di erent components.
4.3.1 Header les
Header les which are commonly used when programming in one of these lan-
guages are available; for example <stdio.h> for C programs or <iostream.h>
for C++ programs.
Speci c header les for writing programs using the ChorusOS application pro-
gramming interface (API) are also available. A simpli ed view of the organiza-
tion of the header les tree is shown in the following gure:
include/

chorus/ CC/ posix/ stdc/

chorus.h exec/ ipc/ am/ alloc.h iostream.h sys/ math.h stdio.h setjmp.h

chExec.h chTime.h chIpc.h afexec.h filio.h sockio.h

The make environment de nes the location of these les. Header les will be
automatically searched in the directories if the standard tools are used and the
environment of the user is correctly initialized. This allows you to use include
macros like #include <ipc/ipc.h> or #include <math.h>.
 include/chorus: this directory is the root of a subtree where the kernel's
APIs and the actor manager are exported. This directory contains a num-
ber of header les (with .h extension) and subdirectories. Each subdirectory
corresponds to the interface of a particular feature. For example, the ipc subdi-
rectory contains the header les associated with the IPC feature, the am subdi-
rectory corresponds to the header le associated to the interface with the actor
manager. These les contain the de nitions of constants and types, and the
prototypes of the functions of the corresponding API. A program will use the
directive #include <chorus.h> to include the most commonly needed de ni-
tions;
 include/stdc: this directory is the root of a subtree where the standard C,
the mathematical C and the console input/output APIs are exported. For ex-
porting the standard input/output API, the directive <stdio.h> will be used;
 include/CC: this directory is the root of a subtree where the GNU C++ API
is exported;
66 Chapter 4. Producing ChorusOS applications
 include/posix: this directory is the root of a subtree where various POSIX
APIs are exported (standard C, math, input/output, Posix Network, Sun RPC
and some BSD calls).
4.3.2 Symbolic constants
Appendix B lists the constants which are de ned in the kernel and the ac-
tor manager interfaces and which can be used when developing applications.
Constants which are related to the kernel API have a K_ pre x. For example
K MYACTOR may be used for naming the current actor and K MYSELF may be
used for naming the currently executing thread. Constants associated to the
actor manager API have AFX or AM pre xes. The constants used for other
APIs are classic constants (for example NULL, O RDONLY, . . . ) used in BSD or
Posix systems.
4.3.3 Prede ned types
Many types are de ned in the header les. The types associated with the
kernel API have the Kn pre x. For example KnCap is the type of a capability,
KnLapDesc is the type of lap descriptor, and so on. Appendix C lists these
prede ned types.
4.3.4 Primitives
A list of the primitives provided by the di erent features, and which will be
presented in this book, is given in appendix D.
4.3.5 Errors
Services provided by the ChorusOS kernel return either the K OK value in case
of success or a negative error code; error codes have a symbolic name starting
with K_E.
The strSysError function, whose prototype is
char *strSysError(int);
returns a speci c C character string when called with one of the negative errors
codes as a parameter. The di erent symbolic constants corresponding to possi-
ble errors returned when invoking a kernel service are listed in appendix A.
Kernel services related to private data return positive error codes. The corre-
sponding symbolic constants, which are pre xed by PD_E and prede ned in the
chPd.h le, are listed in appendix A.
Regular BSD or Posix services return errors using the errno convention and
positive error codes. The perror Posix function (void perror(char *)) may
be used to display a standard message corresponding to an error code.
4.4. General principles 67
4.4 General principles
4.4.1 The libraries
The functional organization of the various libraries is illustrated by the following
tree:

nucleus/ os/

lib/ am/ iom/

classix/ libm/ embedded/ stubs/ pd/ stdc/ amlibc.a disk.a ufs.a

libsys.s.a libsys.u.a libm.a libebd.s.a libebd.u.a kern.a pd.s.a pd.u.a libC.a stdc.s.a stdc.u.a

If the standard tools are used and the correct rules are applied in the Imake le,
the link editor extracts the di erent functions into the corresponding libraries.
The lib/classix subdirectory contains two fundamental libraries:
 libcx.u.a is a library used by user actors for accessing the kernel services. It
contains one object le for each primitive which can be called by a user actor;
 libcx.s.a is a library used by supervisor actors for accessing the kernel ser-
vices.
The crt subdirectory contains:
 the crth.u.o and crt0.o object les which de ne the startup routine (pro-
gram entry point) of user actors;
 the crth.s.o and crt0.o object les which de ne the startup routine of su-
pervisor actors.
The last subdirectory we mention is the CC. It contains the libC.a library (stan-
dard C++ library). This library provides support for C++ applications with a
complete and thread-safe library package. It can be used by user and supervisor
actors.
It is important to note that some libraries cannot be used in all con gurations
of the system. For example, the lib/classix/librpc.a (corresponding to the
sun RPC API) cannot be used within a basic environment.
4.4.2 Entry point
Before executing, the program entry point must be set to start , which calls
main once the libraries have been initialized.
68 Chapter 4. Producing ChorusOS applications
4.4.3 Supervisor binaries
Since all supervisor actors share the same address space (the supervisor address
space), the binary codes generated for these actors are relocatable. The choice
of the nal link is left to the con gurator for embedded actors and to the actor
manager for dynamically loaded actors.
4.4.4 User binaries
The binaries produced for user actors are also relocatable, the link address of
such an actor and the maximum size of its address space are board dependent
and all user actors are linked at the same address (see chapter 7).
4.5 The production's environment
4.5.1 The make environment
This is de ned by a le which contains the de nitions of variables and rules
for producing applications from source codes written in C, C++ and assembly
language. As previously mentioned, when using the GNU compilation chain
this le is gcc-devsys.mk.
The following variables are de ned:
 CFLAGS and CXXFLAGS:de ne the options for compiling C or C++ applications.
Di erent options are available:
- WARN: has two possible settings WARN ON (default value) and WARN OFF.
When the WARN ON value is set, the compiler produces warning messages;
- DEBUG: has two possible settings DEBUG ON and DEBUG OFF (default value).
When DEBUG ON is set, information for a debugging application is pro-
duced;
- PROF: has two possible settings PROF ON and PROF OFF (default value).
When PROF ON is set, extra code for producing information suitable for
the analysis of the program is generated;
- OPT: has two possible settings OPT ON (default value) and OPT OFF. When
OPT ON is set, code optimization is performed;
 INCLUDES speci es the search path for the header les. This variable may be
overloaded at the application level;
 DEPENDS speci es the depend value. Like INCLUDES, this variable may be
overloaded at the application level;
 LD UCRT0, LD SCRT0, LD-CRT1 and LD-SCTN are used to manage the startup
code (crt) of produced object les;
4.5. The production's environment 69
 LD U ACTOR and LD S ACTOR give link information for user and supervisor ac-
tors, respectively;
 CLX U LIBS, CLX S LIBS, EBD U LIBS, EBD S LIBS and CXX LIBS give informa-
tion concerning the libraries for user actors, supervisor actors, embedded user
actors, embedded supervisor actors and C++ libraries.
4.5.2 The imake environment
As shown in the gure in section 4.1, various les de ne the imake environment:
 Imake.tmpl: contains the de nitions of standard variables:
- FAMILY: its value of ix86 or powerpc de nes the target family;
- COMPILER: its value (for example gcc) de nes the compiler to be used;
 Imake.rules: contains the de nition of standard prede ned build rules which
can be used to build an Imake le:
- ActorTarget(prog, objs, options, crt0, libs):
creates a binary, under the name prog, by using the object les corre-
sponding to objs and produced by the C compiler with the speci ed
options and by transmitting the values crt0 and libs to the linker.
This rule is the lowest level one, and is called by other rules to pro-
duce speci c types of actors by using the C compiler;
- UserActorTarget(prog, objs, libs):
creates, under the name prog, a binary corresponding to a user actor
by using the object les produced by the C compiler which corre-
spond to objs, and by transmitting libs to the linker;
- SupActorTarget(prog, objs, libs):
creates a binary corresponding to a supervisor actor (object les
produced by the C compiler);
- EmbeddedUserActorTarget(prog, objs, libs):
creates a binary for an embedded user actor (object les produced
by the C compiler);
- EmbeddedSupActorTarget(prog, objs, libs):
creates a binary for an embedded supervisor actor (object les pro-
duced by the C compiler);
- CXXActorTarget(prog, objs, options, crt0, libs):
is the equivalent of ActorTarget for C++ applications;
70 Chapter 4. Producing ChorusOS applications
- CXXUserActorTarget(prog, objs, libs):
creates a binary corresponding to a user actor (object les produced
by the C++ compiler);
- CXXSupActorTarget(prog, objs, libs):
creates a binary corresponding to a supervisor actor (object les
produced by the C++ compiler);
- CXXEmbeddedUserActorTarget(prog, objs, libs):
creates a binary for an embedded user actor (object les produced
by the C++ compiler);
- CXXEmbeddedSupActorTarget(prog, objs, libs):
creates a binary for an embedded supervisor actor (object les pro-
duced by the C++ compiler);
- BuiltinDriver(prog, objs, libs):
creates a ChorusOS driver;
- BspProgTarget(prog, entry, objs, libs):
creates a BSP program;
- LibraryTarget(lib, objs):
adds the object les to the lib library;
- MakeDir(dir):
creates the directory dir;
- Depend(srcs):
computes the dependencies of the les speci ed by srcs and adds
them to the Make le by calling the makedepend command;
 Project.tmpl: contains the de nition of variables and rules speci c to a
project. If no such le exits in the source directory for the application, an
empty le is used;
 Package.rules: contains the rules allowing a binary distribution of a com-
ponent to be built:
- DistActor(prog, dir):
creates the dir directory and copies into it the prog component;
- DistFile( le, dir):
creates the dir directory and copies into it the le le;
4.5. The production's environment 71
- DistProgram(prog, dir):
creates the dir directory and copies into it the prog program;
- DistRenFile( le, newFile, dir):
creates the dir directory and copies into it the le le with the newFile
name;
- DistLibrary(lib, dir):
creates the dir directory and copies into it the lib library.
4.5.3 Producing binaries
4.5.3.1 The ChorusOSMkMf command
The ChorusOSMkMf utility builds a Makefile from the Imakefile according to
the imake le environment; it uses the standard imake utility and the values of
variables and the rules de ned in the con guration les.
The general form of the command is as follows:
ChorusOSMkMf con g- le [-s source-dir] [-b build-dir] [-d dist-dir]
where:
 con g- le is the directory containing the ChorusOS binary. In all our exam-
ples, we will use the value of the BUILD DIR variable of our environment (its
value is build dir where the system image was built);
 source-dir is the directory containing the source codes. The default value is
the current directory (where the command is called);
 build-dir is the directory where the binary code will be produced. The default
value is the current directory (where the command is called);
 dist-dir is the directory where the binary delivered is stored.
4.5.3.2 Producing the dependencies
The second step consists of generating the make dependencies; this is done by
calling the standard make command with the depend argument. Hence, building
the Makefile with all dependencies consists of the following sequence:
--> ChorusOSMkMf $BUILD_DIR
--> make depend
...............
-->
In order to use the Imake le facility consistently, modi cations should be ap-
plied to the Imakefile rather than to the generated Makefile. The Makefile
le may be regenerated by issuing the command make Makefile once the
Imakefile has been updated.
72 Chapter 4. Producing ChorusOS applications
4.5.3.3 Producing the binary
Once the Makefile and the dependencies have been created, the binary is built
by calling the make command.
4.5.4 Examples
4.5.4.1 Producing examples
In the examples given in 3.8.4, we used di erent binaries corresponding to very
simple applications (trivialAct and noDelay1). Here we show how they were
produced.
The two applications were developed in the sources/utilities directory of our
workspace: we rst created, for each application, a subdirectory which contains
the source le of the application (there is only one le for these applications)
and the corresponding Imake le. In these Imake les (and all Imake les we will
create later), we de ne a number of variables:
 SRCS : its value is a list of source les (with extensions .c for C source les,
.C for C++ les or .s for assembly les). The di erent names are separated
by spaces;
 OBJS : its value is a list of object les (with .o extension).
--> cd $CHORUS/sources/utilities/trivialAct/C
--> cat trivialAct.c
#include <stdio.h>
main(int argc, char *argv[ ]) {
int ind;
printf("Hello! I am the trivial actor\n");
if(argc == 1)
return;
printf("I have %d parameters\n", argc - 1);
for(ind = 1; ind < argc; ind ++)
printf(" ==> %s\n", argv[ind]);
}
-->
We use the following Imakefile located in the same directory as the source le
to produce the executable binaries in the two modes:
--> cat Imakefile
SRCS=trivialAct.c
OBJS=trivialAct.o
UserActorTarget(trivialAct_u, $(OBJS), )
SupActorTarget(trivialAct_s.r, $(OBJS), )
Depend($(SRCS))
-->
We then execute, in the directory containing the Imakefile, the sequence of
4.5. The production's environment 73
commands which produce the Make le, compute the dependencies and produce
the executable binaries:
--> ChorusOSMkMf $BUILD_DIR
--> ls -l Makefile
-rw-r--r-- 1 jmr users 3625 dec 14 18:33 Makefile
--> make depend
depend trivialAct.c
--> ls -l Makefile
-rw-r--r-- 1 jmr users 5029 dec 14 18:34 Makefile
--> make
cc trivialAct.c
ld trivialAct_u
ld trivialAct_s.r
-->

The two binaries have been created in the current directory and are installed
in the bin86 directory (mounted as /mnt on the ix86 target systems):
--> ls triv*
trivialAct.c trivialAct.o trivialAct_s.r trivialAct_u
--> mv trivialAct_s.r trivialAct_u $CHORUS/bin86
-->

The same procedure is followed to produce the binaries corresponding to the


noDelay1 application:

--> cd $CHORUS/sources/utilities/noDelay1
--> cat noDelay1.c
#include <chorus.h>
main( ) {
threadDelay(K_NOTIMEOUT); /* the calling thread sleeps for ever */
}
--> cat Imakefile
SRCS=noDelay1.c
OBJS=noDelay1.o
UserActorTarget(noDelay1_u, $(OBJS), )
SupActorTarget(noDelay1_s.r, $(OBJS), )
Depend($(SRCS))
--> ChorusOSMkMf $BUILD_DIR
--> make depend
depend noDelay1.c
--> make
cc noDelay1.c
ld noDelay1_u
ld noDelay1_s.r
--> mv noDelay1_u noDelay1_s.r $CHORUS/bin86
-->
74 Chapter 4. Producing ChorusOS applications
4.5.4.2 Building a library and using it
a) Creating a library
We now develop, in the sources/utilities/lib directory, two functions that
we will use frequently in examples in the book and build a library to contain
them (later, we will add other functions to that library). These functions per-
form input/output operations on capabilities. A capability corresponds to the
KnCap type which is de ned as:
struct KnCap {
KnUniqueId ui; /* unique identifier of the capability */
KnKey key; /* key of the capability */
};
typedef struct KnCap KnCap;

A capability encapsulates a unique identi er ( eld ui) whose type is KnUniqueId


and is de ned as:
struct KnUniqueId {
unsigned uiHead : 32; /* head of the ui */
unsigned uiTail : 32; /* tail of the ui */
};
typedef struct KnUniqueId KnUniqueId;

and a key (key eld) which has two sub elds keyHead and keyTail which are
also de ned as elds of 32 bits. This type is de ned as:
struct KnKey {
unsigned keyHead : 32; /* head of the key */
unsigned keyTail : 32; /* tail of the key */
};
typedef struct KnKey KnKey;
The library utilities.a that we will install in the $CHORUS/lib86 directory
will rst contain the two following functions:
 void fprintCap(FILE *, char *, KnCap *) which writes to the le asso-
ciated to the FILE pointer (it may be stdout or stderr if the actor has a le
context, or it may have been obtained explicitly through a call to the fopen
standard function) the message given as second parameter and the capability
pointed to by the third one as a four hexadecimal 4-bytes number;
 void readCap(char **, KnCap *) which extracts from the rst parameter
a capability: this parameter should contain the representation of a capability as
4 hexadecimal ASCII numbers. The resulting capability is stored at the address
given as a second argument.
First of all, a header le (named utilities.h) which corresponds to the library
is built, it contains the prototypes of the two functions of the library. This le
4.5. The production's environment 75
is located in the include directory of our workspace:
--> cat $CHORUS/include/utilities.h
#include <stdio.h>
#include <chorus.h>
void fprintCap(FILE *, char *, KnCap *);
void readCap(char **, KnCap *);
-->

The source code of these two functions corresponds to the two following les in
the sources/utilities/lib directory:
--> cd $CHORUS/sources/utilities/lib
--> cat fprintCap.c
#include "utilities.h"
void fprintCap(FILE *stm, char *message, KnCap *actorCap) {
fprintf(stm, "%s: ", message);
fprintf(stm, "%x %x %x %x\n", actorCap -> ui.uiHead, actorCap -> ui.uiTail,
actorCap -> key.keyHead, actorCap -> key.keyTail);
fflush(stm);
}
--> cat readCap.c
#include "utilities.h"
void readCap(char *argv[ ], KnCap *actorCap) {
int aux;
sscanf(argv[0], "%x", &aux); actorCap -> ui.uiHead = aux;
sscanf(argv[1], "%x", &aux); actorCap -> ui.uiTail = aux;
sscanf(argv[2], "%x", &aux); actorCap -> key.keyHead = aux;
sscanf(argv[3], "%x", &aux); actorCap -> key.keyTail = aux;
}
-->

The following Imakefile located in the same directory as the source les is
used to build the library:
--> cat Imakefile
INCLUDES=-I$(CHORUS)/include
SRCS=readCap.c fprintCap.c
OBJS=readCap.o fprintCap.o
LibraryTarget(libutilities.a, $(OBJS))
Depend($(SRCS))
-->

It is worth noting that the INCLUDES variable of the make environment has
been overloaded. The $(CHORUS)/include is added to the list of directories to
search for header les as it contains the utilities.h header le that we use.
We generate the corresponding Makefile and the dependencies as we did earlier,
and nally we use the make command:
76 Chapter 4. Producing ChorusOS applications
--> ChorusOSMkMf $BUILD_DIR; make depend; make
depend readCap.c fprintCap.c
cc readCap.c
cc fprintCap.c
ar libutilities.a
-->

We check that the library contains the two object les and install it in the lib86
directory:
--> ar tv libutilities.a
rw-r--r-- 1001/ 101 1064 dec 14 19:05 1999 readCap.o
rw-r--r-- 1001/ 101 1004 dec 14 19:05 1999 fprintCap.o
--> mv libutilities.a $CHORUS/lib86
-->

Remark. In the codes of the fprintCap and readCap functions given above,
we use the structure of a capability. A version of these functions which is
transparent to this structure is given below:
--> cat fprintCap2.c
#include "utilities.h"
void fprintCap(FILE *stm, char *message, KnCap *actorCap) {
int ind, *array;
fprintf(stm, "%s:", message);
array = (int *) actorCap;
for (ind = 0; ind < sizeof(KnCap) / sizeof(int); ind ++)
fprintf(stm, " %x", array[ind]);
putc('\n', stm);
fflush(stm);
}
--> cat readCap2.c
#include "utilities.h"
void readCap(char *argv[ ], KnCap *actorCap) {
int ind, *array;
array = (int *) actorCap;
for (ind = 0; ind < sizeof(KnCap) / sizeof(int); ind ++)
sscanf(argv[ind + 1], " %x", array + ind);
}
-->

b) Using a library
The noDelay2 application used here is a modi ed version of the noDelay1 ap-
plication which uses some basic services provided by the kernel:
 the capability of the current actor is retrieved by calling the actorSelf prim-
itive and is printed by calling the fprintCap function of our library (therefore,
the corresponding header le is included in the source le);
4.5. The production's environment 77
 then the local identi er of the main thread of the actor is obtained by calling
threadSelf and it is printed;
 nally, a message is printed before the main thread enters the sleeping state
after calling threadDelay with the K NOTIMEOUT value of the parameter.
The corresponding C program is the following:
--> cat $CHORUS/sources/utilities/noDelay2/noDelay2.c
#include "utilities.h"
KnCap actorCap;
main( ) {
actorSelf(&actorCap);
fprintCap(stdout, "Actor's capability", &actorCap);
printf("Thread's identification: %d\n", threadSelf( ));
printf("Hello. I'm gonna sleep\n");
fflush(stdout); /* useful if stdout has been redirected */
threadDelay(K_NOTIMEOUT);
printf("After threadDelay\n"); /* later, if thread is aborted */
}

We now give the corresponding Imakefile which is located in the same directory
as the application. Essentially, it gives information for retrieving the header le
through the INCLUDES variable and requests the linkage of the library in the
production rules for the user and supervisor binaries:
--> cat $CHORUS/sources/utilities/noDelay2/Imakefile
INCLUDES=-I$(CHORUS)/include
SRCS=noDelay2.c
OBJS=noDelay2.o
UserActorTarget(noDelay2_u, $(OBJS), $(CHORUS)/lib86/libutilities.a)
SupActorTarget(noDelay2_s.r, $(OBJS), $(CHORUS)/lib86/libutilities.a)
Depend($(SRCS))
-->
The binaries are now produced and installed:
--> cd $CHORUS/sources/utilities/noDelay2
--> ChorusOSMkMf $BUILD_DIR; make depend; make
[
........ more lines ]
.......
--> mv noDelay2_s.r noDelay2_u $CHORUS/bin86
--> ls $CHORUS/bin86/noDelay2*
/ens/jmr/CHORUS/bin86/noDelay2_s.r
/ens/jmr/CHORUS/bin86/noDelay2_u
-->
A user actor is dynamically created and loaded to execute the application:
--> neon-n noDelay2_u &
[1] 6316
--> started aid = 2
78 Chapter 4. Producing ChorusOS applications
Actor's capability: 20000017 869da80a 2 1
Thread's identification: 9
Hello. I'm gonna sleep
and the actor is killed:
--> rsh neon akill 2
-->

4.6 Dynamic actors


4.6.1 Overview
ChorusOS r4 provides di erent types of libraries:
 the static libraries whose names are suxed by .a. This type of library is a
collection of object les. When an application uses library functions, the linker
just includes the di erent object les extracted from the library in the resulting
object le;
 the dynamic libraries whose names are suxed by .so. Unlike static libraries,
they are linked at run time and not at build time, and are not included in the
binary le. They can be used by user and supervisor actors (not by boot ac-
tors). Actors which use dynamic libraries are called dynamic actors.
ChorusOS r4 provides a component, called the runtime linker, to support dy-
namic linking which may occur:
- at actor start-up: the runtime linker loads and links the libraries when
loading the actor;
- during the execution of an actor: by calling the dynamic linking API (the
dlopen function), an actor can explicitly load and link a library during
its execution.
4.6.2 Building a dynamic library
4.6.2.1 The DynamicLibraryTarget macro
The imake environment de nes a macro to build dynamic libraries:
DynamicLibraryTarget(dlName, objs, staticLibs, dynLibs, dlDeps, opt)
where:
 dlName is the name of the dynamic library to be built. This name must have
.so as a sux;
 objs is the list of the object les used to build the libraries;
 staticLibs is a list of static libraries which will be statically linked;
 dlDeps is a list of dynamic libraries' dependencies;
 dynLibs is a list of dynamic libraries which must be dynamically linked with
the library when it is loaded; it de nes a list of dependencies;
4.6. Dynamic actors 79
 opt is a list of linker options. The following options are generally used:
-soname=name : sets the internal name of the library to be built;
-rpath list: de nes a list of directories (separated by a semicolon ";")
which are added to the library search path.
4.6.2.2 Example
To create a dynamic version of the libutilities library containing the readCap
and fprintCap functions, the corresponding Imakefile is de ned as:
--> cat Imakefile
INCLUDES=-I$(CHORUS)/include
SRCS=readCap.c fprintCap.c
DynamicLibraryTarget(libutilities.so, readCap.o fprintCap.o, , , ,\
-soname=libutilities.so)
Depend($(SRCS))

The library is built by the usual sequence:


--> ChorusOSMkMf $BUILD_DIR
--> make depend
depend readCap.c fprintCap.c
--> make
ld libutilities.so
--> file libutilities.so
libutilities.so: ELF 32 bits LSB dynamic library 80386

To install the dynamic library in the /lib directory in the root le system
mounted on the neon system:
--> rsh neon mount | grep root
root_device on / (pdevfs)
134.157.168.9:/home/jmr/CHORUS/r4/build/root/ on / (nfs)
--> cp $CHORUS/lib86/libutilities.so /home/jmr/CHORUS/r4/build/root/lib

4.6.3 Using a dynamic library


4.6.3.1 Dynamic link at actor startup
We have just created the binary corresponding to the noDelay2 application (see
4.5.4.2) which will use the dynamic version of the library; the library will be
loaded at actor start-up.
Number of macros are provided in the imake environment: DynamicUserTarget,
DynamicSupTarget, DynamicCXXUserTarget and DynamicCXXSupTarget:
DynamicUserTarget(prog, objs, staticLibs, dynLibs, dlDeps, opt)
Therefore, the Imakefile to build the program using the dynamic library con-
tains the following (the sequence to build the program and to install it is also
executed):
80 Chapter 4. Producing ChorusOS applications
--> cat Imakefile
INCLUDES=-I$(CHORUS)/include
SRCS=noDelay2.c
OBJS=noDelay2.o
DynamicUserTarget(noDelay2.dyn_u, $(OBJS), ,\
$(CHORUS)/lib86/libutilities.so, ,\
-rpath /libraries)
Depend($(RCS))
--> ChorusOSMkMf $BUILD_DIR; make depend; make
............
--> mv noDelay2.dyn_u $CHORUS/bin86
-->

4.6.3.2 Dynamic programming


Within the previous approach, the source program of an application using a
dynamic library is not changed. We describe here an alternative where on-
demand binding of objects is performed. An application will perform speci c
operations for that purpose:
2 locate a dynamic object and add it to the address space of an application
by calling
#include <cx/dlfcn.h>
void *dlopen(
const char *pathname,
int mode
);

This call returns an opaque handle to the calling thread which any thread of
the actor may use on subsequent calls to dlsym and dlclose. Here pathname
is the path name of the shared object to be opened. Di erent values of mode
are recognized (see the man command for more details). Here, we just mention
the RTLD NOW value which is e ectively a request that all necessary relocations
are performed when the object is rst loaded.
2 get the address of a symbol (typically a function) in a shared object by
calling
#include <cx/dlfcn.h>
void *dlsym(
void *handle,
const char *name
);
4.6. Dynamic actors 81
This call returns the address of the symbol whose name is given in the object
referenced by the handle. Typically, the returned value is a pointer to the
function whose name is given.
2 close a shared objet by calling
#include <cx/dlfcn.h>
int dlclose(void *handle);

This call dissociates a shared object, previously opened by calling dlopen, from
the current actor. Upon successful completion, 0 is returned. If the object could
not be closed, or if handle does not refer to an open object, the call returns a
non-zero value (the dlerror function may be called to get a null-terminated
character string describing this error).
2 the next example is a version of the noDelay application as a dynamic
program:
--> cat $CHORUS/sources/utilities/noDelay2/dynProg/noDelay2.c
#include "utilities.h"
#include <cx/dlfcn.h>
KnCap actorCap;
void *handle;
void (*function)( );

main( ) {
handle = dlopen("libutilities.so", RTLD_NOW);
if(handle == NULL) {
printf("Can't open libutilities.so\n");
exit(2); }
function = (void (*)( )) dlsym(handle, "badFunction");
if(function == NULL)
printf("Can't find badFunction\n");
function = (void (*)( )) dlsym(handle, "fprintCap");
if(function == NULL) {
printf("Can't find fprintCap\n");
exit(2); }
actorSelf(&actorCap);
(*function) (stdout, "Actor's capability", &actorCap);
printf("Thread's identification: %d\n", threadSelf( ));
printf("Hello. I'm gonna sleep\n");
fflush(stdout);
threadDelay(K_NOTIMEOUT);
printf("After threadDelay\n");
}

The Imake le to build the binary is similar to the one used in 4.6.3.1. The
program is installed in the bin86 as noDelay2.dynProg u.
82 Chapter 4. Producing ChorusOS applications
4.6.4 Running a dynamic program
First of all, the archive that has been booted on the target must include the
DYNAMIC LIB feature.
--> rsh neon arun cs -lM
.............. DL ...............
-->

If we try to execute a program using a dynamic library, we get an error as the


dynamic library cannot be located:
--> neon noDelay2.dyn_u
C_INIT: arun of /mnt/noDelay2.dyn_u failed - No such file or directory
--> noDelay2.dynProg_u
C_INIT: arun of /mnt/noDelay2.dynProg_u failed - No such file or directory
-->

A solution is to use the LD LIBRARY PATH environment variable to specify the


directories where the dynamic libraries are installed (here it is /lib):
--> rsh neon setenv LD_LIBRARY_PATH /lib
--> neon-n noDelay2.dyn_u&
[1] 15292
--> started aid = 2
Actor's capability: 2000001c 869da80a 2 1
Thread's identification: 14
Hello. I'm gonna sleep

-2-> neon-n noDelay2.dynProg_u&


[2] 15298
-2-> started aid = 22
Can't find badFunction
Actor's capability: 2000001e 869da80a 16 1
Thread's identification: 9
Hello. I'm gonna sleep
Chapter 5
General services
We describe here general concepts related to names of objects in ChorusOS:
unique identi ers, capabilities and local identi ers. We also present general
primitives for building them, accessing the environment and reading and writing
to the system console.
5.1 Naming in ChorusOS
Here we present in more detail the di erent levels of naming that are used in
ChorusOS which were mentioned in previous sections.
5.1.1 Unique identi ers
5.1.1.1 De nition and characteristics
The ChorusOS naming scheme is based on unique identi ers which have a global
scope in a given domain. These identi ers are basically used to name commu-
nication ports. They are global and location-independent; this means that a
user will use the same identi er for naming a port wherever it is located and,
moreover, if the corresponding object migrates from one place to another (it
is possible for a port), its unique identi er will not change. They are unique
in time and space: this means unique identi ers are guaranteed to be unique
within a ChorusOS domain over its lifetime. Unique identi ers may be freely
exchanged between actors as part of messages.
In the current version of the system, unique identi ers are 64 bits long and
comprise the following components:
 the type of the resource identi ed by the unique identi er (see 5.1.1.6);
 a local timestamp on the creation site which is guaranteed to be unique within
the lifetime of the local system;
 the identi er of the site where the unique identi er was created.
83
84 Chapter 5. General services
5.1.1.2 The KnUniqueId type
As seen in 4.5.4.2, from the programmer's point of view a unique identi er
corresponds to the KnUniqueId type which has two elds uiHead and uiTail
de ned as elds of 32 bits:
struct KnUniqueId {
unsigned uiHead: 32;
unsigned uiTail: 32;
};
typedef struct KnUniqueId KnUniqueId;

Various functions can be used to manipulate unique identi ers, these are de-
tailed below.
5.1.1.3 Clearing a unique identi er
Given a pointer ui to a unique identi er, the function call
#include <ipc/chId.h>
int uiClear(KnUniqueId *ui);

initializes the unique identi er pointed to by ui with a standard null value. It is


implemented as a macro and thus, if an illegal address is passed as an argument,
an exception will occur in the calling thread, no error will be returned.
A call to the function, which is also de ned as a macro,
#include <ipc/chId.h>
int uiValid(KnUniqueId *ui);

returns 1 if the unique identi er pointed to by ui is not equal to the null unique
identi er and 0 otherwise.
5.1.1.4 Comparing unique identi ers
The function call (de ned as a macro)
#include <ipc/chId.h>
int uiEqual(
KnUniqueId *ui1,
KnUniqueId *ui2
);

returns 1 if the unique identi ers pointed to by ui1 and ui2 are equal and 0
otherwise.
5.1. Naming in ChorusOS 85
5.1.1.5 Sites and unique identi ers
The function call
#include <ipc/chId.h>
int uiSite(
KnUniqueId *ui,
unsigned long site
);

returns the value of the unique identi er which is prede ned for the site whose
site number is site.
The function call
#include <ipc/chId.h>
int uiGetSite(KnUniqueId *ui);

returns the site information contained in the unique identi er pointed to by ui.
The function call
#include <ipc/chId.h>
int uiIsLocal(KnUniqueId *ui);

checks whether the given unique identi er is located on the current site at
present. The function returns 1 if this is case and 0 otherwise.
5.1.1.6 Building a unique identi er
A unique identi er corresponding to a particular type of object may be built
and returned at the ui address by calling the function
#include <ipc/chId.h>
int uiBuild(
KnUniqueId *ui,
unsigned int type,
unsigned int site,
unsigned int head,
unsigned int tail
);

where
 type is the type of the unique identi er with the following recognized values:
- K UIPORT: the unique identi er will be recognized as a unique identi er
for a port for the IPC module. This type of identi er can be used when
calling portDeclare (cf. 13.3.1.4);
86 Chapter 5. General services
- K UIGROUP: the unique identi er will be recognized as a unique identi er
for a port group by the IPC module and will be usable when calling
grpPortInsert (cf. 13.3.2.3);
- K UISITE: the unique identi er of a site;
 site is a site number which, if not null, will be used as a hint by the kernel
when building the unique identi er;
 head and tail represent the stamp of the unique identi er: tail is a 32 bits
value and head is a 13 bits value.
The function returns K OK upon success, and K EFAULT if any data is outside
the address space of the current actor. The call uiSite (ui, site) is equivalent
to uiBuild (ui, K UISITE, site, 0, 0).
5.1.2 Capabilities
5.1.2.1 De nition and characteristics
Capabilities are the most general type of identi ers: a capability encapsulates
a unique identi er and provides means for protecting and restricting access to
objects. Capabilities are associated to actors, groups of ports and segments.
If we consider a server managing di erent objects, a common use of capabilities
consists in identifying the di erent objects managed by the server by capabilities
built with a common unique identi er (typically identifying a port where the
server should be contacted), and a key managed by the server which identi es
the object and possibly de nes speci c protection attributes of the object.
5.1.2.2 The KnCap type
From the programmer's point of view a capability corresponds to the KnCap
type de ned as
struct KnCap {
KnUniqueId ui;
KnKey key;
};
typedef struct KnCap KnCap;

As already stated, a capability encapsulates a unique identi er ( eld ui) as


described above and a key ( eld key) which has two sub elds keyHead and
keyTail which are de ned as elds of 32 bits. This type is de ned as:
struct KnKey {
unsigned keyHead: 32;
unsigned keyTail: 32;
};
typedef struct KnKey KnKey;
5.1. Naming in ChorusOS 87
The structure of a capability can be illustrated as follows:
KnCap

KnKey key
KnUniqueId ui

unsigned uiHead:32 unsigned uiTail:32 unsigned keyHead:32 unsigned keyTail:32

In 4.5.4.2, we have built and inserted in a utilities.a library functions for


printing and reading capabilities.
5.1.3 Local identi ers
Local resources of an actor (for example threads or ports) are named by using
local identi ers: the scope of a local identi er is limited to the actor where the
object is located. To designate a thread from an actor di erent from the actor
owning the thread, it is necessary to add to the local identi er of the thread
the capability of its actor. As we have seen on the results of the cs command,
local identi ers are also used for naming actors locally on a given site.
5.1.4 Getting the local site number
Every site is identi ed by an integer. When calling the function
#include <ipc/chId.h>
int uiLocalSite(int *site);

the integer identifying the local site is returned at address site. The K OK value
is returned except if site is out of the address space (in that case K EFAULT is
returned).
5.1.5 Example
In the next example, the application uses di erent functions of the API we
presented in preceding sections.
--> cat $CHORUS/sources/utilities/general/general.c
#include <chorus.h>
KnUniqueId ui;
int site;
main( ) {
uiLocalSite(&site);
printf("Local site number: 0x%x [%d]\n", site, site);
uiBuild (&ui, K_UISITE, site, 0, 0); /* uiSite(&ui, site); */
printf("Predefined UI: 0x%x 0x%x\n", ui.uiHead, ui.uiTail);
88 Chapter 5. General services
portUi(&ui, K_DEFAULTPORT);
printf("Default port's UI: 0x%x 0x%x\n", ui.uiHead, ui.uiTail);
if(uiIsLocal(&ui))
printf(" It is recognized\n");
else
printf(" It is not recognized\n");
site = uiGetSite(&ui);
printf("information on site extracted from 0x%x 0x%x: 0x%x\n",
ui.uiHead, ui.uiTail, site);
}
--> neon general_u
started aid = 2
Local site number: 0x869da80a [-2036488182]
Predefined UI: 0x0 0x869da80a
Default port's UI: 0x2000001c 0x869da80a
It is recognized
information on site extracted from 0x2000001c 0x869da80a: 0x869da80a
-->

5.2 Getting information about ChorusOS


As seen in 2.4, when building and con guring a system, one may set tunable
values, environment variables and select or reject kernel features. This section
describes how an application may retrieve this information.
The <exec/chModules.h> le de nes, for each module of the ChorusOS:
 an integer constant K_MODULE_moduleName (for example K MODULE SEM for
the module corresponding to the SEM feature);
 a character string K_MODULE_moduleName_NAME corresponding to the ocial
name of the module (for example K_MODULE_EXEC_NAME is prede ned to the
string "CORE" which is the ocial name associated with the executive).
5.2.1 Getting a tunable parameter
For each module, a header le de nes a symbolic constant for each tunable
parameter of the module whose value can be retrieved.
For example, the <exec/chExec.h> header le contains the de nitions of several
constants:
- K_GETCONF_ACTOR_MAX identi es the kern.exec.maxActorNumber param-
eter;
- K_GETCONF_THREAD_MAX identi es the kern.exec.maxThreadNumber pa-
rameter.
5.2. Getting information about ChorusOS 89
The function call
#include <exec/chExec.h>
int sysGetConf(
const char *moduleName,
int parameter,
int *ptrValue
);

gets the value associated with the tunable parameter identi ed by parameter
in the module whose name is moduleName. The parameter argument may be
K GETCONF VERSION when querying for the version of the module. In the SEW
2.0 product, version numbers are all set to 1 except for the IPC feature which
is set to 1 when it is local, and to 2 when IPC REMOTE is enabled.
If the call succeeds, *ptrValue is the value of the parameter returned and K OK
is returned by the function. Otherwise, a negative error code is returned:
return value error type
K ENOTAVAILABLE non existing module in the system
K EFAULT address out of address space
Hence, in order to check whether a feature has been con gured within the sys-
tem, an application may call the function and check whether it returns K OK
(the feature is con gured) or K ENOTAVAILABLE (it is not).
In the next example, we extract the values of the tunable parameters corre-
sponding to the maximum numbers of actors and threads the system can man-
age and the scheduling module which is used:
--> cat $CHORUS/sources/utilities/getConf/getConf.c
#include <stdio.h>
#include <exec/chModules.h>
#include <sched/chSched.h>
#include <sched/chFifo.h>
main( ) {
int result, value;
result = sysGetConf(K_MODULE_EXEC_NAME, K_GETCONF_ACTOR_MAX, &value);
printf("Maximum number of actors: %d\n", value);
result = sysGetConf(K_MODULE_EXEC_NAME, K_GETCONF_THREAD_MAX, &value);
printf("Maximum number of threads: %d\n", value);
result = sysGetConf(K_MODULE_SCHED_NAME, K_GETCONF_VERSION, &value);
printf("Scheduler's version: %d\n", value);
}
We rst execute the application on the system neon where the default scheduling
module (based on fo policy) is active:
90 Chapter 5. General services
--> neon getConf_u
started aid = 2
Maximum number of actors: 64
Maximum number of threads: 128
Scheduler's version: 1
-->
Now we have generated a system where the ROUND ROBIN feature has been ac-
tivated, have loaded it on the carbon system where we execute the getConf
application :
--> carbon getConf_u
started aid = 2
Maximum number of actors: 64
Maximum number of threads: 128
Scheduler's version: 2 (= a di erent scheduler  a di erent result
-->

5.2.2 Accessing the environment


As seen in 2.4, the system environment is de ned by a set of dynamic parameters
whose value is a character string.
5.2.2.1 Getting a value: sysGetEnv
The function call
#include <exec/chEnv.h>
int sysGetEnv(
const char *envName,
char *ptrValue,
unsigned int *ptrSize
);

is a request for the value of the Chorus con guration environment variable whose
name is envName. The ptrValue argument is the address to which the result
is to be written. On input, *ptrSize should be initialized to the size in bytes
of memory which has been allocated to ptrValue, and in output it gives the
number of bytes of the value (including the null terminating character). If the
call succeeds K OK is returned, and otherwise a negative error code is returned:
return value error type
K EFAIL unde ned variable
K ENOMEM not enough space allocated to ptrValue and
*ptrSize is the necessary space
K EFAULT address out of address space
5.2. Getting information about ChorusOS 91
5.2.2.2 Setting a value: sysSetEnv
The function call
#include <exec/chEnv.h>
int sysSetEnv(
const char *envName,
const char *envValue
);

replaces the current value of the envName variable of the ChorusOS environment
by envValue if this variable is de ned. If the variable is not de ned, the variable
and the value are inserted in the environment. The envValue argument must
point to a null terminated string. The call returns K OK if it succeeds, otherwise
a negative error code is returned:
return value error type
K EINVAL length of envName is 0
K ENOMEM not enough free space (maximum size is limited to a tunable
parameter [default is 1K])
K EFAULT address out of address space

5.2.2.3 Deleting a value: sysUnsetEnv


The function call
#include <exec/chEnv.h>
int sysUnsetEnv(const char *envName);

deletes the envName variable from the environment of the ChorusOS.


K OK is returned if the call succeeds, otherwise a negative error code is returned.

return value error type


K EINVAL envName does not correspond to a de ned variable
K EFAULT address out of address space

5.2.2.4 Example
--> cat $CHORUS/sources/utilities/envir/envir.c
#include <stdio.h>
#include <chorus.h>
main( ) {
char buf[256];
int result, size = 256;
result = sysGetEnv("TARGET", buf, &size);
92 Chapter 5. General services
if (result != K_OK)
fprintf(stderr, "sysGetEnv(TARGET): %s\n", strSysError(result));
else
printf("value of TARGET: %s\n", buf);
size = 256;
result = sysGetEnv("HOST", buf, &size);
if (result != K_OK)
fprintf(stderr, "sysGetEnv(HOST): %s\n", strSysError(result));
else
printf("value of HOST: %s\n", buf);
result = sysSetEnv("VAR", "Just to see");
if (result != K_OK)
fprintf(stderr, "sysSetEnv(VAR): %s\n", strSysError(result));
size = 256;
result = sysGetEnv("VAR", buf, &size);
if (result != K_OK)
fprintf(stderr, "sysGetEnv(VAR): %s\n", strSysError(result));
else
printf("value of VAR: %s\n", buf);
result = sysUnsetEnv("VAR");
if (result != K_OK)
fprintf(stderr, "sysUnsetEnv(VAR): %s\n", strSysError(result));
result = sysGetEnv("VAR", buf, &size);
if (result != K_OK)
fprintf(stderr, "sysGetEnv(VAR): %s\n", strSysError(result));
else
printf("value of VAR: %s\n", buf);
}

The command is rst executed on a system where the only de ned environment
variable is OS CONF:
--> neon envir_u
started aid = 2
sysGetEnv(TARGET): Transaction failed
sysGetEnv(HOST): Transaction failed
value of VAR: Just to see
sysGetEnv(VAR): Transaction failed
-->

We rst generate a new system where the variables TARGET, and HOST are de-
ned:
--> cd /home/jmr/CHORUS/r4/build
--> configurator -p conf/extended
--> configurator -setenv TARGET=carbon
--> configurator -setenv HOST=fluorine
--> make chorus
[ ]
.... many more lines .....
-->
5.3. Input/output from the system console 93
We then execute the envir application on the carbon machine where this con-
guration has been booted:
--> carbon envir_u
started aid = 2
value of TARGET: carbon
value of HOST: fluorine
value of VAR: Just to see
sysGetEnv(VAR): Transaction failed
-->

5.3 Input/output from the system console


The target system is connected to the host system via a serial line (see 2.5.5)
and this connection is used as system console for the target.
5.3.1 Reading from the system console: sysRead
A call to the primitive
#include <exec/chIo.h>
int sysRead(
void *bu er,
unsigned int nbChar
);
corresponds to a request for reading characters from the system console. If there
is no character to be read, the calling thread is blocked. Otherwise, characters
are read and transferred into the bu er pointed to by bu er, until nbChar-1
characters have been read or a newline character is encountered (this character
will not be transferred in the bu er). The characters that are read are echoed on
the system console. If nbChar is 1, one character is read without being echoed.
The function returns the number of characters that have been read and K EFAULT
if bu er is out of the address space.
5.3.2 Writing to the system console: sysWrite
A call to the primitive
#include <exec/chIo.h>
int sysWrite(
void *bu er,
unsigned int nbChar
);
is a request to write nbChar characters onto the system console from the bu er
pointed to by bu er. The function returns the number of characters that have
been actually written and K EFAULT if bu er is out of the address space.
94 Chapter 5. General services
5.3.3 Polling the system console: sysPoll
The function call
#include <exec/chIo.h>
int sysPoll(void *bu er);

attempts to read one character from the system console into the bu er pointed
to by bu er. If there is no character to be read, the calling thread of the actor
in not blocked. The function returns the number of characters that have been
read (0 or 1) and the K EFAULT value if bu er is out of the address space.
5.3.4 Example
-1-> cat $CHORUS/sources/general/console/console.c
#include <exec/chIo.h>
char bufferIn[256];
char bufferPoll[16];
char *bufferOut = "Message from the system\n";
main( ) {
int result;
result = sysWrite(bufferOut, strlen(bufferOut));
printf("sysWrite --> %d\n", result);
result = sysWrite("Enter new message: ", 19);
result = sysRead(bufferIn, 256);
printf("sysRead --> %d\n", result);
result = sysWrite(bufferIn, 10);
result = sysWrite("\n", 1);
while ((result = sysPoll(bufferPoll)) <= 0) ; /* busy loop */
printf("From first sysPoll: %d\n", result);
while ((result = sysPoll(bufferPoll)) <= 0) ; /* busy loop */
printf("From second sysPoll: %d %c\n", result, bufferPoll[0]);
}
-1-> neon console_u < the console >
started aid = 22
Message from the system
sysWrite --> 24
Enter new message: abcdefghijklmnopq
sysRead --> 17
abcdefghij
From first sysPoll: 1
-2-> rsh neon arun cs -la 22
started aid = 23
ABCD (=
from keyboard
From second sysPoll: 1 A
-1->
error: not a local actor
ChorusOS r4.0.0 Site 0 Time 39m 44
5.4. Rebooting the local system 95
5.4 Rebooting the local system
A thread executing with the supervisor privilege may request a reboot of the
local system by calling the primitive
#include <exec/chPanic.h>
int sysReboot(KnRebootReq *rebootReq);

The KnRebootReq type has a KnRebootMode eld whose value may be


K REBOOT COLD, K REBOOT HOT or K REBOOT NEW.
An application can only use the K REBOOT COLD value. This call stops all kernel
drivers and initiates a BSP-dependent cold reboot procedure. If the sysReboot
function is called with a NULL argument, the cold reboot is performed.
The function normally does not return. If the calling thread is not running with
the supervisor privilege, the call returns K EPRIV.
Chapter 6
Actors
6.1 Introduction
An actor is the unit of program modularization for applications developed by
users and for subsystems. It encapsulates resources like an address space or
ports for point-to-point communications. It provides a frame for execution of
threads which all share the resources of the actor and therefore de nes a virtual
machine for the user.
For example, after the C INIT actor interprets an arun command, the newly
created c actor encapsulates
 di erent memory regions (for the code, the data and the stack of the main
thread);
 one port for point to point communication (actor's default port);
 one thread (the main thread executing the main function).
An actor is tied to a site and cannot migrate to another one. All the threads it
contains run on that site and they all share the address space of the actor they
belong to.
When an actor is created, the kernel allocates a descriptor in an internal pool
where characteristics for the actor (such as capability or status) are recorded.
When it is rst created at the lowest level an actor is essentially empty; it may
contain one private port (default port) if the IPC feature is active. It contains
no thread and its address space is empty, these are dynamically created after-
wards.
Actors may be divided into di erent categories according to the nature of their
address space and to their privilege:
97
98 Chapter 6. Actors
 user actors have their own address space. Some user actors may have addi-
tional privileges: they are said to be trusted and are also called system actors.
Threads of trusted actors get access to certain additional services (for example
they may create system or supervisor actors). User actors require appropriate
support from the memory features, that is, the VIRTUAL ADDRESS SPACE feature
must be set;
 supervisor actors share the supervisor address space. This means that if
the actor is badly written and accesses addresses outside its address space, data
of other supervisor actors or data of the kernel itself may be corrupted. These
actors are, by de nition, trusted and their threads will execute with supervisor
privilege. The following diagram summarizes these di erent types of actors:
actor

supervisor user
actor : K SUPACTOR
c actor: AFX SUPERVISOR SPACE

trusted (system) not trusted


actor : K SYSTEMACTOR actor : K USERACTOR
c actor: AFX USER SPACE, AFX TRUSTED c actor: AFX USER SPACE
As previously mentioned, each instance of the ChorusOS system includes the
executive module. It provides a programming interface for manipulating actors
at a very low level, resources have to be attached to the actors by calling speci c
primitives.
Besides the basic service o ered by the executive, we have mentioned that the
ACTOR EXTENDED MNGT feature provides extended management functions such as
dynamic loading and control of actors. It also provides a support for POSIX
functions such as le input/output operations. A speci c system actor, called
the actor manager (AM), may be requested for creating actors; the actor man-
ager uses the service of the kernel for creating the actors. The actor manager
also provides other services:
 memory regions are created for the actor, and executable code and data are
loaded;
 the actor has a le context attributed via the IOM actor;
 a thread is created in the actor: it executes the main function of the code and
therefore is often called its main thread.
As previously mentioned, these actors are called c actors (or extended ac-
tors). We will see that actors that have been created without using the services
of the actor manager may be promoted later on (see 6.4.2).
The actor in the context of which execution occurs is called the current actor.
6.2. Actors attributes 99
6.2 Actors attributes
6.2.1 Identifying an actor
Each actor is identi ed by a capability and has a local identi er on the system
it belongs to. In the case of c actors, local identi ers may be used to identify
the actors in speci c functions. In the interface, a pointer to the capability of
the target actor is generally used as a parameter and the K MYACTOR symbolic
constant may be used in that context when the current actor is concerned.
6.2.1.1 Capability of an actor
The function call
#include <exec/chExec.h>
int actorSelf(KnCap *actorCap);

returns the capability of the current actor into the structure pointed to by ac-
torCap, which must have been previously allocated.
The function returns K OK except if an address error is detected (actorCap point-
ing outside the address space of the actor), in which case the K EFAULT value is
returned.
6.2.1.2 Symbolic name
A symbolic name may be associated to an actor. This name is a string whose
length is at most K ACTORNAMEMAX (default value is 16). The old name of an
actor may be retrieved and/or a new name may be associated to an actor by
calling the function
#include <exec/chExec.h>
int actorName(
KnCap *actorCap,
VmAddr oldName,
VmAddr newName
);

A NULL value for any argument disables the corresponding operation.


Upon success the function returns K OK, otherwise a negative error code is re-
turned (K EINVAL or K EUNKNOWN for a non valid capability, or K EFAULT for an
illegal address).
100 Chapter 6. Actors
6.2.1.3 The case of c actors
As with any actor, a c actor is identi ed in the system by a capability. As a
c actor, it also has a local identi er which is used in speci c APIs provided by
the ACTOR EXTENDED MNGT feature. As we will see later, these identi ers are
returned by afexec primitives used to create new c actors.
The function call
#include <am/afexec.h>
int agetId(void);

returns the current c actor's local identi er.


Given the local identi er aid of a c actor on a given site, a call to
#include <am/afexec.h>
int acap(
int aid,
KnCap *cactorCap
);

returns at cactorCap the capability of the corresponding c actor on the site. Like
the cs command, this function gets the capabilities of actors. This information
should be con dential, as once it is known, services like deleting or changing
attributes of the corresponding actors can be accessed. Thus, its use is restricted
to trusted c actors.
The function returns 0 if it is successful and -1 otherwise. In that case, the
external errno variable is set to a value corresponding to the error:
value of errno error type
EINVAL aid is not a valid c actor's local identi er
EFAULT cactorCap points outside the address space of the c actor
ESRCH aid is not the identi er of a c actor on the site
EPERM the calling c actor is not trusted

6.2.2 Actors states


6.2.2.1 The basic states
An actor has a global state which indicates whether the threads it contains may
be scheduled or not. This state has two possible values:
 the active state: all threads in the actor may be scheduled;
 the stopped state: no thread in the actor may be scheduled until the actor
becomes active.
6.2. Actors attributes 101
Depending the way it is created, an actor may be in one of these two states
upon creation. Furthermore, its state may be changed by a call to one of the
two functions actorStart or actorStop.
The following diagram describes the changes to the state of an actor
actorStart

Stopped Active
actorStop

K STOPPED K ACTIVE
actorCreate

6.2.2.2 Changing the basic state of an actor


The function call
#include <exec/chExec.h>
int actorStop(KnCap *actorCap);

stops the actor whose capability is pointed to by actorCap. All threads of the
actor are stopped. The e ect may not be immediate on the actor's threads
currently engaged in a system call, but the system guarantees that there will
not be a return from this call until the actor is activated.
The function call
#include <exec/chExec.h>
int actorStart(KnCap *actorCap);

reactivates the actor whose capability is pointed to by actorCap. After that


call, all the ready threads of the actor are active (they become competitors for
scheduling except if they are waiting for an external event).
These two calls return K OK when they are successful, otherwise a negative error
code is returned (K EINVAL, K EUNKNOWN or K EFAULT).
6.2.3 Actors privilege
An actor may be a USER actor, a SYSTEM actor or a SUPERVISOR actor.
The type of an actor de nes its privilege. It is de ned by the KnActorPriv-
ilege type which has three de ned values:
 K SUPACTOR: the actor is a SUPERVISOR actor;
 K SYSTEMACTOR the actor is SYSTEM actor (trusted user actor);
 K USERACTOR: the actor is a USER actor.
102 Chapter 6. Actors
The function call
#include <exec/chExec.h>
int actorPrivilege(
KnCap *actorCap,
KnActorPrivilege *oldPriv,
KnActorPrivilege *newPriv
);

gets the current privilege of the actor whose capability is pointed to by actorCap
at the address oldPriv if it is not NULL. Furthermore a supervisor thread or a
thread belonging to a SYSTEM actor may change the privilege of the actor to
the value pointed to by newPriv If it is not NULL. It is only possible to transform
a USER actor into a SYSTEM actor and vice-versa.

6.2.4 Example
In the following example, a c actor is created and loaded with the command
arun with argument charactActor u. This actor writes some of its attributes
to its standard output before setting itself into the stopped state. Another
actor is created for reactivating the stopped actor by executing the actorStart
program. The same task is performed successively for all three types of actor.
-1-> cat $CHORUS/sources/actors/charactActor/charactActor.c
#include "utilities.h"
#include <exec/chExec.h>
KnCap actorCap;
KnActorPrivilege privilege;
int status, localId;
char oldName[256]; /* for getting actor's old name */
char *newName = "NAME's EXAMPLE"; /* actor's new name */

main( ) {
printf("Hello. I'm the new actor\n");
(void) actorSelf(&actorCap); /* getting actor's capability */
/* writing actor's capability */
fprintCap(stdout, "My capability is", &actorCap);
if((localId = agetId( )) != -1) /* getting the local identifier */
printf("I am a c_actor and my local identifier is %d\n", localId);
else
perror("agetId");
/* getting the privilege without modifying it */
actorPrivilege(K_MYACTOR, &privilege, NULL);
6.2. Actors attributes 103
switch(privilege) {
case K_SUPACTOR : printf("I'm a supervisor actor\n"); break;
case K_SYSTEMACTOR : printf("I'm a system actor\n"); break;
case K_USERACTOR : printf("I'm a user actor\n");
}
/* getting symbolic name and changing it */
actorName(K_MYACTOR, oldName, newName);
printf("My old name was: %s\n", oldName);
printf("I'm going to stop ....\n");
actorStop(K_MYACTOR); /* stopping the actor */
printf("Back to the active world !!!\n"); /* after being activated */
}
-1-> neon charactActor_u
started aid = 23
Hello. I'm the new actor
My capability is: 20000073 869da80a 17 2d3
I am a c_actor and my local identifier is 23
I'm a user actor
My old name was: charactActor_u
I'm going to stop ....

Information about the actor locally identi ed by 23 is retrieved by using the


cs command (from a di erent shell on the Solaris host indicated by a di erent
prompt):
-2-> rsh neon arun cs -la 23
started aid = 22
ChorusOS r4.0.0 Site 0 Time 22h 0m 58
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000073 869da80a 00000017 000002d3 0023 USER STOPPED 001 NAME's EXAMPLE
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0008 140 00000000 00000000 ff6330 0- 1- 0 main
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
20000073 869da80a def no fac0ac 0 20000073 869da80a
START SIZE OFFSET ALLOC OPTIONS
fffed000 00006000 00000000 00006000 WR
ffff4000 00001000 00000000 00001000 WR
ffff5000 0000a000 00000000 0000a000 EX
-2->

The stopped actor is restarted by using the following actorStart application


which receives the capability of the target actor as an argument:
-2-> cat $CHORUS/sources/actors/actorStart/actorStart.c
#include "utilities.h"
main(int argc, char *argv[ ]) {
KnCap actorCap;
int rep;
104 Chapter 6. Actors
if(argc != 5) {
fprintf(stderr, "Bad usage ...\n");
exit(1);
}
readCap(argv + 1, &actorCap); /* decoding target actor's capability */
rep = actorStart(&actorCap); /* reactivating target actor */
if (rep < 0)
fprintf(stderr, "actorStart failed: %s\n", strSysError(rep));
}
-2-> neon actorStart_u 20000073 869da80a 17 2d3
started aid = 22
-2->

We can check that the stopped actor is restarted, the message relating to the
printf call appears on the corresponding screen:
Back to the active world !!! (= message from the restarted actor
-1->

We now create a supervisor actor on the same system by requesting the ex-
ecution of the supervisor version (the use of the -S option is not necessary,
information in the binary le is sucient for detecting that a supervisor has to
be created).
-1-> neon charactActor_s.r
started aid = 23
Hello. I'm the new actor
My capability is: 20000076 869da80a 17 2eb
I am a c_actor and my local identifier is 23
I'm a supervisor actor
My old name was: charactActor_s.
I'm going to stop ....

By using the cs command, we also get the attributes of this actor:


-2-> rsh neon arun cs -la 23
started aid = 22
ChorusOS r4.0.0 Site 0 Time 22h 3m 44
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000076 869da80a 00000017 000002eb 0023 SUP STOPPED 001 NAME's EXAMPLE
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0008 140 00000000 00000000 ff6330 0- 1- 0 main
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
20000076 869da80a def no fac0ac 0 20000076 869da80a
START SIZE OFFSET ALLOC OPTIONS
7bbf1000 00002000 00000000 00002000 WR SU
7bbf6000 00001000 00000000 00001000 WR SU
7bbf7000 0000a000 00000000 0000a000 EX SU
-2->
6.3. Getting status of actors on the site 105
6.3 Getting status of actors on the site
6.3.1 Getting status of all actors on the site
The KnActorStat type de ned as
typedef struct KnActorStat {
KnCap asCap; /* capability */
KnActorStatus asStatus; /* K_STOPPED or K_ACTIVE */
} KnActorStat;
is used to retrieve information on the currently known actors through a call to
the function
#include <exec/chExec.h>
int actorStat(
unsigned int ags,
KnActorStat *stat,
unsigned int bu Size
);

which gets the status of all the actors created on the current site and writes
the information about every actor (capability and status) at the address stat in
the address space of the calling actor. This type of call is restricted to threads
running with SUPERVISOR privilege or whose actor is a SYSTEM actor.
The bu Size argument gives the size (expressed as a number of bytes) of the
bu er allocated at the address stat. The only value of ags which is currently
valid is 0. If the call succeeds, the function returns the number of actors existing
on the site. If it fails, a negative value is returned:
- K EFAULT: an argument is outside the address space of the actor;
- K EPRIV: the calling thread is not allowed to get the information (this call
is restricted to trusted entities).
In the actorStat application whose source is given below, the list of actors
running on the site is retrieved by calling actorStat and for each actor, its
capability and its status are displayed (they are directly extracted from the
corresponding entry in the bu er). For every actor, its privilege and its symbolic
name are retrieved by calling actorPrivilege and actorName.
-2-> cat $CHORUS/sources/actors/actorStat/actorStat.c
#include "utilities.h"
#include <exec/chExec.h>
#define N_ACTORS 50
106 Chapter 6. Actors
KnActorStat stat[N_ACTORS];
KnActorPrivilege actorPriv;
char name[K_ACTORNAMEMAX + 1];

main( ) {
int result, ind;
result = actorStat(0, stat, sizeof(stat));
if (result < 0) {
fprintf(stderr, "Error on actorStat : %s\n", strSysError(result));
exit(1);
}
printf("There are %d actors on the site\n", result);
for(ind = 0; ind < result; ind ++) {
actorName(&stat[ind].asCap, name, NULL);
actorPrivilege(&stat[ind].asCap, &actorPriv, NULL);
printf("%-20s", name);
printf("%-8s", (stat[ind].asStatus == K_ACTIVE) ?
"ACTIVE" : "STOPPED");
printf("%-5s", (actorPriv == K_SUPACTOR) ? "SUP" : "USER");
fprintCap(stdout, "capability", &stat[ind].asCap);
}
}
-2-> neon actorStat_u
started aid = 22
Error on actorStat : Privilege violation (=
calling thread is not trusted
-2-> neon actorStat_s.r
started aid = 22
There are 22 actors on the site
kern ACTIVE SUP capability: 20000001 869da80a 1 1
D_pci ACTIVE SUP capability: 20000003 869da80a 3 0
D_pcienum ACTIVE SUP capability: 20000004 869da80a 4 4
[
..... more lines ] ....
AM ACTIVE SUP capability: 20000013 869da80a 13 0
ADMIN ACTIVE SUP capability: 20000014 869da80a 14 14
C_INIT ACTIVE SUP capability: 20000015 869da80a 15 1
actorStat_s.r ACTIVE SUP capability: 2000007f 869da80a 16 0
NAME's EXAMPLE STOPPED SUP capability: 20000076 869da80a 17 2eb
-2->

The last two actors in these results are the current user actor executing the
actorStat application and the stopped supervisor actor we created in the pre-
vious section.
6.3.2 Getting status of all c actors on the site
For every c actor created on the site, the actor manager (AM) owns a descriptor
of this actor. This descriptor is the astatEntry type and has the following elds:
6.3. Getting status of actors on the site 107
int astatFlags; /* actor's status */
int astatAid; /* actor's local identi er */
int astatUid; /* actor's user Id */
char astatName[ ]; /* actor's symbolic name */
The function call
#include <am/astat.h>
int astat(
astatEntry *entries,
int maxEntries
);
gets the entries for to the c actors created on the local site.
At most maxEntries entries will be lled by the function and the return value
is the number of c actors on the site.
The ag ASTAT DEBUGGED CACTOR is used to indicate that the actor is being
debugged.
Upon success, the call returns the number of c actors that are currently active
on the site. If an error occurs, -1 is returned and errno is set to EFAULT due
to a value of entries being outside the address space.
The astat application whose code is given below displays information concern-
ing all c actors known on the local site:
-2-> cat $CHORUS/sources/actors/astat/astat.c
#include "utilities.h"
#include <am/astat.h>
#define N_ACTORS 50
astatEntry stat[N_ACTORS];
char name[K_ACTORNAMEMAX + 1];
main( ) {
int nbAct, result, ind;
KnCap capa;
nbAct = astat(stat, N_ACTORS);
if (nbAct == -1) {
perror("astat");
exit(1);
}
printf("There are %d c_actors on the site\n", nbAct);
for(ind = 0; ind < nbAct; ind ++) {
printf("Local id: %2d ", stat[ind].astatAid);
result = acap(stat[ind].astatAid, &capa);
if (result != -1) {
actorName(&capa, name, NULL);
printf("%-20s", name);
fprintCap(stdout, "capability", &capa);
}
108 Chapter 6. Actors
else
putchar('\n');
}
}
-2-> neon astat_s.r
There are 4 c_actors on the site
Local id: 22 astat_s.r capability: 20000080 869da80a 16 1
Local id: 23 NAME's EXAMPLE capability: 20000076 869da80a 17 2eb
Local id: 21 C_INIT capability: 20000015 869da80a 15 1
Local id: 20 ADMIN capability: 20000014 869da80a 14 14
-2->

We now kill the stopped supervisor c actor by using the akill command before
executing the astat application once again:
-2-> rsh neon akill 23
-2-> neon astat_s.r
started aid = 23 (= id 23 is reused for a new actor
There are 3 c_actors on the site
Local id: 23 astat_s.r capability: 20000081 869da80a 17 0
Local id: 21 C_INIT capability: 20000015 869da80a 15 1
Local id: 20 ADMIN capability: 20000014 869da80a 14 14
-2->

6.4 Creating actors


6.4.1 Introduction
We have seen how an actor could be created through the arun command and
therefore by the actor manager (we called this actor a c actor). An actor can
also be dynamically created by any other actor by using the APIs provided by
the executive and the ACTOR EXTENDED MNGT feature.
This can be done in the following ways:
 by spawning an actor by special calls to one of the afexec functions which
invokes the actor manager. These calls successively create, load a new actor
and start a thread. The created actor is a c actor and it has a le context;
 by rst creating a new empty c actor by a call to the acreate primitive. Later
on a memory context will be loaded for that actor by calling aload. Finally,
the actor will be started by calling astart;
 by directly creating a new actor without involving the actor manager through
a call to the primitive actorCreate of the executive API; the new actor is an
empty actor. Other functions of the API may be applied to the actor afterwards.
6.4. Creating actors 109
The following diagram summarizes these di erent possibilities (U designates a
user actor or a user c actor and S a supervisor actor or a supervisor c actor):
U arun U

afexec user and user and


actorCreate acreate system system
c actors actors user
address
space
supervisor
S
afexec C INIT address
acreate Supervisor Supervisor space
S AM c actors actors

actorCreate actorCreate

CHORUS Kernel

6.4.2 Promoting an actor to the status of c actor


Actors created by the actorCreate call have no le context and are not known
by the actor manager. Any such actor which invokes a function of the actor
manager interface (for example agetId, one of the afexec functions for creating
a c actor or a le operation like open) will automatically be promoted to the
status of c actor. When promoted, an actor is given a default le context
consisting of a root and a current directories set to /, but no open les.

6.4.3 High level creation of c actors


6.4.3.1 Speci c attributes of a c actor
A c actor is a unique actor: besides the general attributes we have presented
(capability, state, . . . ), it has speci c attributes de ning its le context which
may be set at creation time (open les, current directory and credentials).
The AcParam type packs some attributes of a c actor into a unique object. It
is used when calling the afexec and acreate primitives to describe attributes
of the c actor whose creation is requested. It has the following elds:
 int acSite: release 4.0 does not support the creation of remote actors and
this eld is ignored (warning: the AM MYSITE symbolic constant which was gen-
erally used to create actors on the local site is no longer de ned);
110 Chapter 6. Actors
 int acFlags: the options of the new actor are built using bitwise operators
and the following constants:
- AFX USER SPACE: if the ag AFX TRUSTED is also set, the c actor will be
a system actor and will be trusted. Otherwise, the actor will be a user
actor and will not be trusted;
- AFX SUPERVISOR SPACE: the new c actor will be a supervisor actor (and
will be trusted);
- AFX DEBUGGEE: the new c actor will be started in debugged mode;
 const char *acStdin: path name for standard input;
 const char *acStdout: path name for standard output;
 const char *acStderr: path name for standard error output;
 const char *acCurdir: path name for current directory;
 const char *acRootdir: pathname for root directory;
 const am cred t *acCred: credentials for the new actor which de ne the
user and the group owning the actor. The am cred t type is de ned as:
cx_uid_t cr_uid; /* c_actor's user ID */
cx_gid_t cr_gid; /* c_actor's group ID */
unsigned short cr_ngroups; /* number of groups in cr_groups */
cx_gid_t cr_groups[ ]; /* supplementary group list */
The credentials of a c actor may be retrieved or modi ed by a trusted c actor
by calling the function
#include <am/acred.h>
int acred(
const KnCap *cactorCap,
am cred t *oldCred,
const am cred t *newCred
);

A call returns 0 when successful and -1 otherwise (errno will be set to ESRCH,
EPERM or EFAULT).

6.4.3.2 Spawning a c actor: the afexec primitives


This is the easiest way of creating an actor: not only is the actor created but a
memory context is created and loaded (code and data) and a thread is created
and started (the main thread of the actor). This thread has some attributes
which are de ned by tunable parameters of the actor manager:
6.4. Creating actors 111
- its scheduling class is de ned by the am.afexec.schedclass con guration
parameter (by default it corresponds to the FIFO scheduling class, see
8.3.2.1);
- its priority is de ned by the am.afexec.schedprio con guration param-
eter;
- it executes in a stack which is automatically allocated and whose size is
de ned by the am.afexec.userstacksize con guration parameter.
Once created, this thread executes a startup routine and calls the main function
of a program which is de ned as
int main(argc, char *argv[ ], char *arge[ ])

The di erent parameters of the function are interpreted the same way as when
creating Unix processes or loading a new program in an existing process:
- argc is the arguments count (at least one);
- argv is an array of character pointers corresponding to the parameters;
- arge is an array of character pointers corresponding to the environment
of the actor.
Six primitives are available for requesting the actor manager (AM); they look
like exec Unix system calls. The names of these primitives have a common
afexec pre x and di er as follows:
 by the way they receive the arguments for the main function. It is a list ended
by a NULL pointer for afexecl, afexecle and afexeclp and it is a vector for
afexecv, afexecve and afexecvp;
 by the fact that the le to load is searched (afexeclp and afexecvp) using
the PATH variable in the current environment of C INIT;
 by the fact that a new environment is explicitly provided (afexecle and
afexecve) or inherited from the caller.
Their general form is
int afexecxy (char *pathName, KnCap *actorCap, AcParam *param, ...)

For each of these primitives,


 pathName is the name of an executable le which will be loaded;
 param points to an AcParam object describing characteristics of the new
c actor. If some char * elds of the object pointed to by param have a NULL
value, their values are inherited from the corresponding values in the calling
actor.
112 Chapter 6. Actors
The value of the acFlags eld of the object pointed to by param de nes the
type of the new actor as speci ed in 6.4.3.1.
 if the call succeeds, the capability of the new actor is written at the address
actorCap and the call returns the new c actor's local identi er. If the call fails,
-1 is returned and the value of the errno is set to indicate the error:
value of errno interpretation
EPERM calling thread not allowed to create a privileged actor
ENOENT invalid pathname (non existent component)
ENOTDIR non existent directory in the pathname
EACCES search permission denied for a component of pathname
ENOEXEC bad format for the le
EFAULT invalid address for the parameters
EINVAL inconsistent attribute ags
EIO I/O error when reading the le
ENAMETOOLONG name of component of pathName or total name too long
E2BIG total length of arguments is too long
ENXIO the le is not an ordinary le
We now present these di erent primitives.
a) The afexecv primitive
By a call to the primitive afexecv a new c actor will be created whose main
thread will execute the main function of the program loaded from the pathName
le. The arguments of the function are passed as an array of character pointers
ended by a NULL pointer. The prototype of the function is given below:
#include <am/afexec.h>
int afexecv(
const char *pathName,
KnCap *actorCap,
const AcParam *param,
const char *const *argv
);

It is important to note that there is no search for the le when this primitive is
used; the pathname is either absolute or is relative to the current directory.
In the following example, a c actor is created to execute the trivialAct pro-
gram. The command will be run with:
 a rst argument specifying the type of the new actor: u for a user actor, t for
a trusted user actor (system actor) and s for a supervisor actor;
 a second argument specifying the pathname of the le to be loaded and exe-
cuted by the new actor;
6.4. Creating actors 113
 other parameters used as parameters for the main function executed by the
main thread of the new actor. These parameters are used to build a vector of
parameters.
--> cat $CHORUS/sources/actors/afexecv/afexecv.c
#include "utilities.h"
#include <am/afexec.h>
AcParam newActorAttrib; /* attributes of the new c_actor */
char *newActorParam[10]; /* to build the vector of parameters */
KnCap newActorCap; /* new actor's capability */

main(int argc, char *argv[ ], char **arge) {


int ind, result;
newActorAttrib.acSite = 0; /* field ignored: new actor on the local site */
/* type of the new c_actor as deduced from the first argument */
switch(argv[1][0]) {
case 't' : newActorAttrib.acFlags = AFX_USER_SPACE|AFX_TRUSTED;
break;
case 's' : newActorAttrib.acFlags = AFX_SUPERVISOR_SPACE;
break;
default : newActorAttrib.acFlags = AFX_USER_SPACE;
}
/* building arguments' vector */
for(ind = 2; (ind < 9) && (ind < argc); ind ++)
newActorParam[ind - 2] = argv[ind];
/* creating the new actor by spawning the current one */
result = afexecv(argv[2], &newActorCap, &newActorAttrib, newActorParam);
if(result == -1) {
perror("error on afexecv");
exit(2);
}
printf("A new actor has been created \n");
printf(" Its local identifier is %d\n", result);
fprintCap(stdout, " Its capability is", &newActorCap);
}
-->

In the rst call, the pathname used is trivialAct u; as there is no search for
the le, the call to afexecv fails since the le is not referenced in the current
directory (which is the root of the mounted le system):
--> neon afexecv_u u trivialAct_u aaaa bbbb
started aid = 23
error on afexecv: No such file or directory
-->

By passing /mnt/trivialAct u as pathname, (we assume that a mount oper-


114 Chapter 6. Actors
ation has associated the /mnt directory of the target system to the directory
where our binaries are located), the call succeeds:
--> neon afexecv_u u /mnt/trivialAct_u aaaa bbbb
started aid = 23
A new actor has been created
Its local identifier is 22
Its capability is: 20000084 869da80a 16 333
Hello! I am the trivial actor
I have 2 parameters
==> aaaa
==> bbbb
-->

When trying to create a system or a supervisor actor, the call also fails if the
calling actor is a user actor; this type of actor can only be created by a system
or a supervisor actor:
--> neon afexecv_u t /mnt/trivialAct_u aaaa bbbb
started aid = 22
error on afexecv: Not owner
--> rsh neon arun -T /mnt/afexecv_u t /mnt/trivialAct_u xx yy
started aid = 22
A new actor has been created
Its local identifier is 23
Its capability is: 20000087 869da80a 17 34f
Hello! I am the trivial actor
I have 2 parameters
==> xx
==> yy
--> rsh neon arun -T /mnt/afexecv_u s /mnt/trivialAct_s.r A B
started aid = 23
A new actor has been created
Its local identifier is 22
Its capability is: 20000089 869da80a 16 360
Hello! I am the trivial actor
I have 2 parameters
==> A
==> B
-->

b) The afexecvp primitive


The afexecvp primitive is quite similar to the general afexecv primitive we
just presented: it has the same arguments. The only di erence is that there is
a search for the pathname given to be loaded in the set of directories de ned
by the value of the PATH variable in the environment of the current actor.
6.4. Creating actors 115
The prototype of the primitive is de ned as
#include <am/afexec.h>
int afexecvp(
const char *pathName,
KnCap *actorCap,
const AcParam *param,
const char *const *argv
);

In the afexecvp example given below we just changed the call to afexecv
afexecv(argv[2], &newActorCap, &newActorAttrib, newActorParam)

into its equivalent using afexecvp:


afexecvp(argv[2], &newActorCap, &newActorAttrib, newActorParam)

We also add /mnt into the list of directories de ned by the PATH variable of the
C INIT environment.
--> rsh neon env | grep PATH
PATH=/bin
--> rsh neon setenv PATH /bin:/mnt
--> rsh neon env | grep PATH
PATH=/bin:/mnt
--> neon afexecvp_u u trivialAct_u AAAA BB
started aid = 22
A new actor has been created
Its local identifier is 23
Its capability is: 2000008b 869da80a 17 36c
Hello! I am the trivial actor
I have 2 parameters
==> AAAA
==> BB
-->

As the /mnt directory now belongs to the list of directories de ned by the vari-
able PATH, the call to afexecvp succeeds when giving trivialAct u as the
pathname of the le to load (we used the same property when running the com-
mand with arun without specifying the absolute path of the le afexecvp u).
c) The afexecve primitive
When using afexecv or afexecvp, the new actor inherits the environment of
the calling actor. By using the afexecve primitive, it is possible to de ne a new
environment for the new actor. The new actor will not inherit any variables
from the environment of the calling actor. This environment is presented as an
array of strings having the form variable=value terminated by a NULL entry.
116 Chapter 6. Actors
The prototype of the function is de ned as
#include <am/afexec.h>
int afexecve(
const char *pathName,
KnCap *actorCap,
const AcParam *param,
const char *const *argv,
const char *const *arge
);

We now give the source of a simple example using this primitive:


--> cat $CHORUS/sources/actors/afexecve/afexecve.c
#include <stdio.h>
#include <am/afexec.h>
char *newActorEnv[3], /* for the environment */
*newActorParam[2]; /* for the arguments */
AcParam newActorAttrib;
KnCap newActorCap;
int result;
main(int argc, char *argv[ ], char **arge) {
char **ptr = arge;
fprintf(stderr, "Actor %d -> Environment:\n", agetId( ) );
while(*ptr != NULL)
fprintf(stderr, "%s\n", *ptr++);
if(argc == 1) /* if no parameter : exit */
exit(0);
newActorParam[0] = argv[0];
newActorParam[1] = NULL;
newActorAttrib.acSite = 0;
newActorAttrib.acFlags = AFX_USER_SPACE;
result = afexecvp(argv[0], &newActorCap, &newActorAttrib, newActorParam);
if(result == -1) {
perror("error on afexecvp");
exit(1); }
fprintf(stderr, "New actor %d created by actor %d with afexecvp\n",
result, agetId( ));
newActorEnv[0] = "PATH=/";
newActorEnv[1] = "X=10";
newActorEnv[2] = NULL;
result = afexecve("/mnt/afexecve_u",
&newActorCap, &newActorAttrib, newActorParam, newActorEnv);
if(result == -1) {
perror("error on afexecve");
exit(1); }
6.4. Creating actors 117
fprintf(stderr, "New actor %d created by actor %d with afexecve\n",
result, agetId( ));
}
--> neon afexecve_u aaaaaa (= )
one argument not to exit immediately
started aid = 23
Actor 23 -> Environment:
PATH=/bin:/mnt
HOST=
TZ=GMT0
New actor 22 created by actor 23 with afexecvp
Actor 22 -> Environment:
PATH=/bin:/mnt
HOST=
TZ=GMT0
New actor 22 created by actor 23 with afexecve
Actor 22 -> Environment:
PATH=/
X=10
-->

d) The afexecl primitive


When using this primitive, the arguments of the main function of the new actor
are individual arguments of the primitive, a NULL argument marks the end of
the enumeration. It can be used if the arguments are known when writing the
source code. Apart from that di erence, this primitive behaves in the same way
as the afexecv primitive.

#include <am/afexec.h>
int afexecl(
const char *pathName,
KnCap *actorCap,
const AcParam *param,
const char *arg0,
...,
const char *argn,
(char *) NULL
);

e) The afexeclp primitive


This primitive behaves in the same way as the afexecvp primitive except that
arguments for the main function to be executed by the main thread of the new
actor are enumerated.
118 Chapter 6. Actors
#include <am/afexec.h>
int afexeclp(
const char *pathName,
KnCap *actorCap,
const AcParam *param,
const char *arg0,
...,
const char *argn,
(char *) NULL
);
f) The afexecle primitive
This primitive behaves in the same way as the afexecve primitive except that
arguments for the main function to be executed by the main thread of the new
actor are enumerated.
#include <am/afexec.h>
int afexecle(
const char *pathName,
KnCap *actorCap,
const AcParam *param,
const char *arg0,
...,
const char *argn,
(char *) NULL,
const char *const *arge
);

6.4.4 Low level operations on c actors


As previously mentioned, afexec primitives perform a sequence of operations:
creating, loading and starting. For each phase, a low level service is provided.
6.4.4.1 Creating an empty c actor: acreate
A call to the acreate function described below is a request to the actor manager
(AM) to create a new c actor according to arguments described in a AcParam
structure. The prototype of the function is:
#include <am/afexec.h>
int acreate(
KnCap *cactorCap,
const AcParam *param
);
6.4. Creating actors 119
The c actor which is created has no memory region and no thread. Furthermore,
it is in a STOPPED state. On the other hand, as a c actor it owns a le context
which is not the case if it is created by calling the actorCreate primitive we
will present in the next section.
If any char * elds of the object pointed to by param have a NULL value, their
values are inherited from the corresponding values in the calling actor.
The value of the acFlags eld of the object pointed to by param de nes the type
of the new actor in the same way as for afexec primitives. When successful,
the function returns the local identi er of the new c actor and the capability of
the new actor is written at the address cactorCap.
If the call fails, the value -1 is returned and the value of the errno variable is
set to indicate the error:
value of errno error type
EPERM calling thread is not allowed to create this type of c actor
ENOENT a component of the pathname does not exist
ENOTDIR a component of the pathname is not a directory
EFAULT address out of address space
EACCES search permission denied for a directory
EINVAL inconsistent attributes ags
ENAMETOOLONG name of component of pathName or total name too long
E2BIG total length of arguments is too long
In the following example, an empty c actor is dynamically created by calling
the acreate function: the type of the actor is given as argument and standard
input/output are inherited from the calling actor:
-1-> cat $CHORUS/sources/actors/acreate/acreate.c
/* usage: acreate_u type *
* with type: u (user) t (trusted) s (supervisor) *
* pathname: standard output */
#include "utilities.h"
#include <am/afexec.h>
KnCap cactorCap; /* capability of the new actor */
AcParam param; /* attributes of the new actor (initialized to 0) */
int result;

main(int argc, char *argv[ ]) {


if(argc < 3) {
fprintf(stderr, "Bad usage ....\n");
exit(1);
}
printf("A new c_actor is going to be created\n");
param.acSite = 0;
120 Chapter 6. Actors
switch(argv[1][0]) {
case 'u' : param.acFlags = AFX_USER_SPACE; break;
case 't' : param.acFlags = AFX_USER_SPACE | AFX_TRUSTED; break;
case 's' : param.acFlags = AFX_SUPERVISOR_SPACE; break;
default : fprintf(stderr, "unknown actor's type\n"); exit(1);
}
result = acreate(&cactorCap, &param);
if(result == -1)
perror("acreate failed");
else {
printf("A new c_actor has been created with local id: %d\n",
result);
fprintCap(stdout, "and capability", &cactorCap); }
}
-1-> neon acreate_u u
started aid = 22
A new c_actor is going to be created
A new c_actor has been created with local id: 23
and capability: 20000091 869da80a 17 0
Attributes of the new c actor are displayed by calling aps and cs commands
(we use a new window) and we can see that
 the new actor has no name yet;
 the new actor has no thread;
 the new actor has no memory region;
 the new actor has one port.
-2-> rsh neon aps
UID AID NAME DBG GROUP
0 23 unknown 0 N/A
0 21 C_INIT 0 N/A
0 20 ADMIN 0 N/A
-2-> rsh neon arun cs -la 23
started aid = 22
ChorusOS r4.0.0 Site 0 Time 23h 7m 16
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000091 869da80a 00000017 00000000 0023 USER STOPPED 000 <no-name>
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
20000091 869da80a def no fac0ac 0 20000091 869da80a
-2->

6.4.4.2 Loading text and data for a c actor: aload


a) The aload primitive
Once an actor has been created with acreate, it is possible to initialize its
text and data regions (and its environment and its parameters) by calling the
function
6.4. Creating actors 121
#include <am/afexec.h>
int aload(
KnCap *cactorCap,
const char *pathName,
const AlParam *ldParam,
KnPc *entryPoint
);

where:
 cactorCap points to the capability of the target actor which must be a c actor.
Usually, aload is the rst event that is applied to an actor that has been created
by acreate. Nevertheless, it can be applied to any c actor as many times as
necessary. If the address space of the actor is not empty, the address range that
is to be used by the target actor must be free and the actor must be in a state
where its address space may be changed;
 if it not NULL, pathName names an ordinary executable le comprising a
header, a text segment and a data segment that will be loaded;
 if it is not NULL and if the operation succeeds, entryPoint will contain the ad-
dress of the entry point in the program as determined by the loader. Sucient
space to store a KnPc object must have been allocated to entryPoint;
 if not NULL, ldParam points to an object corresponding to the packed rep-
resentation of the environment and the parameters which will be copied in a
region of the target actor. This object may be built from given values of argv
and arge with the function alParamBuild as described in c. If it is NULL, the
arguments and environment currently installed are not a ected.
Upon success the call returns 0. Otherwise a value of -1 is returned and errno
is set to indicate the error:
value of errno error type
EPERM calling thread is not allowed to create such a c actor
ENOENT a component of the pathname does not exist
ENOTDIR a component of the pathname is not a directory
EFAULT address out of address space
EACCES search permission denied for a directory
EINVAL inconsistent attributes ags
ENAMETOOLONG name of component of pathName or total name too long
E2BIG total length of arguments is too long
EIO I/O error when reading the le
ENXIO the le is not an ordinary le
ENOEXEC not a proper le
ESRCH cactorCap does not correspond to a valid c actor
ENOMEM address space of the target actor is not free
122 Chapter 6. Actors
b) Example 1
In the following example, a program is loaded for the actor we created with
acreate in the previous section, by using the following application:
-2-> cat $CHORUS/sources/actors/aload1/aload1.c
/* usage: aload1_u uiHead uiTail keyHead keyTail pathname */
#include "utilities.h"
#include <am/afexec.h>
int result;
KnCap cactorCap;
KnPc entryPoint;

main(int argc, char *argv[ ]) {


if(argc != 6) {
fprintf(stderr, "Bad usage ....\n");
exit(1);
}
readCap(argv + 1, &cactorCap);
result = aload(&cactorCap, argv[5], NULL, &entryPoint);
if(result == -1)
perror("aload failed");
else
printf("aload succeeded !!!\n");
}
-2-> neon aload1_u 20000091 869da80a 17 0 noDelay2_u
started aid = 22
aload failed: No such file or directory
-2-> neon aload1_u 20000091 869da80a 17 0 /mnt/noDelay2_u
started aid = 22
aload succeeded !!!
-2-> rsh neon arun cs -la 23
started aid = 22
ChorusOS r4.0.0 Site 0 Time 23h 12m 40
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000091 869da80a 00000017 00000000 0023 USER STOPPED 000 <no-name>
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
20000091 869da80a def no fac0ac 0 20000091 869da80a
START SIZE OFFSET ALLOC OPTIONS
ffff5000 00001000 00000000 00001000 WR
ffff6000 00009000 00000000 00009000 EX
-2->

c) Preparing parameters of the main function


As mentioned previously, the aload function copies a packed representation of
the arguments and of the environment into a region of the created c actor. This
representation is built by calling
6.4. Creating actors 123
#include <am/afexec.h>
void alParamBuild(
void *bu er,
unsigned int bu erSize,
char **argv,
char **arge
);

The call stores, at the address speci ed by bu er, the packed representation of
arguments de ned by argv and environment de ned by arge, bu er may then be
passed as third argument to the aload function. The value of bu erSize gives
the amount of space allocated to this location which must be large enough to
hold the packed representation. By convention, the AlParam object built should
not contain more than ARG MAX bytes.
The functions which follow are used by the startup of the C library to build
the arguments and the environment in the usual representation. The address
where the packed representation of the arguments and environment have been
loaded is retrieved by calling the function
#include <am/afexec.h>
int agetalparam(AlParam **argp);

The function
#include <am/afexec.h>
unsigned int alParamSize(AlParam *alParamp);

returns the size of the packed representation stored at the location alParamp.
Finally, the arguments and environment are retrieved by calling
#include <am/afexec.h>
int alParamUnpack(
AlParam *alParamp,
int *argcp,
char ***argvp
);
d) Example 2
In the following, an empty user c actor is rst created by calling the acreate
primitive. Then, its arguments (deduced from the arguments of the calling
actor) and its environment (identical to the environment of the calling actor)
are packed before calling aload:
124 Chapter 6. Actors
-2-> cat $CHORUS/sources/actors/aload2/aload2.c
/* usage: aload2_u pathname arg1 arg2 ... */
#include <stdio.h>
#include <chorus.h>
#include <am/afexec.h>
#include "utilities.h"
KnCap cactorCap; /* capability of the new actor */
AcParam param; /* attributes of the new actor */
KnPc entryPoint; /* to get the entry point from the loader */
char *pointer;
AlParam *alParamP;
int result;
main(int argc, char *argv[ ], char **arge) {
if(argc < 2) {
fprintf(stderr, "Bad usage ....\n");
exit(1); }
/* create an empty c_actor */
param.acSite = 0;
param.acFlags = AFX_USER_SPACE;
param.acStdout = NULL;
result = acreate(&cactorCap, &param);
if(result == -1)
perror("acreate failed");
else {
printf("A new c_actor has been created with local id: %d\n",
result);
fprintCap(stdout, "and capability", &cactorCap);
}
/* build the packed list of arguments */
pointer = (char *) malloc(0x1000);
alParamP = (AlParam *) pointer;
alParamBuild(alParamP, 0x1000, argv + 1, arge);
result = aload(&cactorCap, argv[1], alParamP, &entryPoint);
if(result == -1)
perror("aload failed");
else
printf("aload succeeded !!!\n");
/* astart(&cactorCap); */
}

We use this application to create an actor and load the trivialAct u with
three arguments:
-2-> neon aload2_u /mnt/trivialAct_u aaa bbbbb cccc
started aid = 22
A new c_actor has been created with local id: 24
and capability: 20000097 869da80a 18 3d3
aload succeeded !!!
6.4. Creating actors 125
6.4.4.3 Activating a c actor: astart
The function call
#include <am/afexec.h>
int astart(KnCap *cactorCap);

creates a thread in the actor whose capability is pointed to by cactorCap. The


thread executes the program segment of the actor at the entryPoint speci ed
at the load operation.
The function returns 0 upon success and -1 otherwise (errno is set to ESRCH
when the capability is not valid).
We use the astart function to activate a thread executing the main function
of the le we just loaded (remember that the standard output of the new actor
has been redirected to a le when acreate was called):
-3-> cat $CHORUS/sources/actors/astart/astart.c
/* usage: astart_u uiHead uiTail keyHead keyTail */
#include "utilities.h"
int result;
KnCap cactorCap;
main(int argc, char *argv[ ]) {
if(argc != 5) {
fprintf(stderr, "Bad usage ....\n");
exit(1);
}
readCap(argv + 1, &cactorCap);
result = astart(&cactorCap);
if(result == -1)
perror("astart failed");
else
printf("astart succeeded !!!\n");
}
-3->

We rst apply this application to the rst actor we created and loaded:
-3-> neon astart_u 20000091 869da80a 17 0
started aid = 22
astart succeeded !!!
-3->

We can see on the screen corresponding to the actor (the rst screen) the dif-
ferent messages that it prints:
Actor's capability: 20000091 869da80a 17 0
Thread's identification: 8
Hello. I'm gonna sleep
126 Chapter 6. Actors
We now kill the actor by using the akill command:
-3-> rsh neon akill 23
-3->

We now apply the astart application with a list of arguments to the second
actor we created and loaded:
-1-> neon astart_u 20000097 869da80a 18 3d3
started aid = 23
astart succeeded !!!
-1->

Hello! I am the trivial actor (= information


I have 3 parameters (= from
==> aaa (= the
==> bbbbb (= started
==> cccc (= actor
-2->

6.4.5 The low level interface: actorCreate


Any actor may invoke the kernel actorCreate primitive to create a new actor.
The actor manager (AM actor) is not invoked; the actor is not a c actor and is
not known to the AM actor. The new actor which is created is empty in the
sense that it has no memory region and no thread and unlike c actors it has no
le context.
This service is provided through the primitive
#include <exec/chExec.h>
int actorCreate(
KnCap *actorInitialCap,
KnCap *actorCap,
KnActorPrivilege privilege,
KnActorStatus status
);

The new actor inherits exception handlers and abort handlers (see 13.2 and 13.3)
from the initializing actor whose capability is pointed to by actorInitialCap. It
is worth noting that the initializing actor may be di erent from the calling actor
even if they are generally the same (argument actorInitialCap set to the value
K MYACTOR).
The privilege argument may be K USERACTOR, K SYSTEMACTOR or K SUPACTOR. A
SYSTEM actor can only be created by a thread running with the SUPERVISOR
6.4. Creating actors 127
privilege or belonging to a SYSTEM actor. Furthermore, the new actor may
be created with the SUPERVISOR privilege: memory space will be allocated
in the supervisor address space and all its threads will run with SUPERVISOR
privilege.
In the status argument the following ags may be set:
 K STOPPED: the actor is created in the stopped state. None of its threads will
be able to run when created until the actor is put into the active state by a call
to actorStart;
 K ACTIVE: the new actor is active.
K OK is returned if the actor was created. Otherwise, a negative error code is
returned:
return value interpretation
K EINVAL inconsistent actor's capability
K EUNKNOWN unknown actor
K EFAULT incorrect address
K ENOMEM system out of resources
K EPRIV non authorized thread for creating a system actor
The following application can be used to create new actors by calling the
actorCreate primitive. The command will be called with two parameters:
 the rst parameter de nes the privilege of the new actor to be created (u for
a user actor, t for a trusted user actor and s for a supervisor actor);
 the second parameter de nes the state of the newly created actor: a for active
and s for stopped.
Before terminating, the c actor which created the actor will display the capa-
bility of the newly created actor:
--> cat $CHORUS/sources/actors/actorCreate/actorCreate.c
#include "utilities.h"
int result;
KnActorPrivilege privilege;
KnActorStatus state;
KnCap capa;
main(int argc, char *argv[ ]) {
if (argc < 3) {
fprintf(stderr, "error ...\n");
exit(0);
}
switch (argv[1][0]) {
case 't' : privilege = K_SYSTEMACTOR; break;
case 's' : privilege = K_SUPACTOR; break;
default : privilege = K_USERACTOR;
}
128 Chapter 6. Actors
state = (argv[2][0] == 's') ? K_STOPPED : K_ACTIVE;
result = actorCreate(K_MYACTOR, &capa, privilege, state);
if(result == K_OK)
fprintCap(stdout, "New actor's capability", &capa);
else
fprintf(stderr, "Error on actorCreate: %s\n", strSysError(result));
}
-->

We use this application to create, on the target neon:


- an active user actor;
- a stopped system (trusted user) actor;
- an active supervisor actor.
--> neon actorCreate_u u a
started aid = 24
New actor's capability: 200000a2 869da80a 17 424
--> neon actorCreate_u t s
Error on actorCreate: Privilege violation (= non trusted caller
--> rsh neon arun -T actorCreate_u t s
started aid = 24
New actor's capability: 200000a5 869da80a 16 0
--> neon actorCreate_s s a
started aid = 24
New actor's capability: 200000a7 869da80a 2 440
-->

We can see from the results of the cs command that the actors exist and are
empty, they only own a port. The results of the aps command show that these
new actors are not known to the actor manager. They cannot be deleted by the
akill command:
--> rsh neon arun cs
started aid = 24
ChorusOS r4.0.0 Site 0 Time 23h 42m 44
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000001 869da80a 00000001 00000001 0001 SUP STARTED 003 kern
200000a7 869da80a 00000002 00000440 0002 SUP STARTED 000 <no-name>
[
........ other actors ] .......
200000a5 869da80a 00000016 00000000 0022 USER STOPPED 000 <no-name>
200000a2 869da80a 00000017 00000424 0023 USER STARTED 000 <no-name>
200000a8 869da80a 00000018 00000001 0024 SUP STARTED 001 cs
--> rsh neon arun cs -la 2
started aid = 24
ChorusOS r4.0.0 Site 0 Time 23h 45m 13
ACTOR-UI KEY LID TYPE STATUS TH# NAME
200000a7 869da80a 00000002 00000440 0002 SUP STARTED 000 <no-name>
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
200000a7 869da80a def no ea9e40 0 200000a7 869da80a
6.5. Deleting actors 129
--> rsh neon arun cs -la 22
started aid = 24
ChorusOS r4.0.0 Site 0 Time 23h 47m 16
ACTOR-UI KEY LID TYPE STATUS TH# NAME
200000a5 869da80a 00000016 00000000 0022 USER STOPPED 000 <no-name>
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
200000a5 869da80a def no fac1e8 0 200000a5 869da80a
--> rsh neon arun cs -la 23
started aid = 24
ChorusOS r4.0.0 Site 0 Time 23h 47m 44
ACTOR-UI KEY LID TYPE STATUS TH# NAME
200000a2 869da80a 00000017 00000424 0023 USER STARTED 000 <no-name>
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
200000a2 869da80a def no fac0ac 0 200000a2 869da80a
--> rsh neon aps
UID AID NAME DBG GROUP
0 21 C_INIT 0 N/A
0 20 ADMIN 0 N/A
--> rsh neon akill 2
C_INIT: akill: 2: cannot get actor capability
-->

6.5 Deleting actors


6.5.1 The low level interface: actorDelete
A call to the primitive
#include <exec/chExec.h>
int actorDelete(KnCap *actorCap);

is a request to delete the actor whose capability is pointed to by actorCap.


As usual, the K MYACTOR value designates the current actor. When an actor
is deleted, all resources it owned are returned to the system; ports, memory
regions and threads are deleted.
The calling actor must run on the same site as the target actor.
If handlers have been associated to traps or exception in the deleted actor, they
must be explicitly disconnected before the call (see chapter 14).
The K OK value is returned by the call if the actor can be deleted. Otherwise, a
negative error code is returned (K EINVAL, K EUNKNOWN or K EFAULT). Note that
when actorDelete is called with K MYACTOR as an argument, there is no return!
The following corresponds to the code of a standard application which requests
the deletion of an actor whose capability is passed as argument:
130 Chapter 6. Actors
--> cat $CHORUS/sources/actors/actorDelete/actorDelete.c
#include "utilities.h"
KnCap actorCap;
int result;
main(int argc, char *argv[ ]) {
readCap(argv + 1, &actorCap);
result = actorDelete(&actorCap);
if (result != K_OK)
fprintf(stderr, "error on actorDelete: %s\n", strSysError(result));
else
printf("Actor has been deleted\n");
}
-->
We use this application to delete the three empty actors we created with the
actorCreate application:
--> neon actorDelete_u 200000a7 869da80a 2 440
started aid = 24
Actor has been deleted
--> neon actorDelete_u 200000a5 869da80a 16 0
started aid = 24
Actor has been deleted
--> neon actorDelete_u 200000a2 869da80a 17 424
started aid = 24
Actor has been deleted
--> neon actorDelete_u 200000a2 869da80a 17 424
started aid = 24
error on actorDelete: Invalid argument
--> rsh neon arun cs | grep "no-name"
started aid = 24
-->

6.5.2 Terminating c actors


A c actor is an actor and therefore it may be deleted by calling the actorDelete
primitive.
Besides this possibility, a c actor will terminate and will be deleted when its
main thread (the thread created when it is loaded and started via arun or one
the afexec primitives) terminates after a normal return. It may also be killed
by the speci c akill call which corresponds to the akill command transmitted
to the C INIT actor.
6.5.2.1 Termination of the main thread
Firstly, the main thread terminates when it returns from the initial call to the
main function. This termination may also occur by an explicit call to one of
the exit or exit Posix functions.
6.5. Deleting actors 131
A call to the POSIX primitive
#include <unistd.h>
int exit(int status);

terminates the calling actor and:


 all open le descriptors are closed;
 all other c actors currently blocked on await or awaits calls for that actor
are awoken and noti ed of the c actor's termination (see 5.6).
The exit function of the standard C library may be used for deleting an actor.
A call to the primitive
#include <stdlib.h>
int exit(int status);

terminates the calling actor and:


 C stream les (open through fopen) are ushed;
 functions registered through the atexit standard function are called in reverse
order;
 cleanup functions are performed;
 a exit(status) call is performed to terminate the c actor.
6.5.2.2 Killing a c actor: akill
A call to the function
#include <am/afexec.h>
int akill(KnCap *actorCap);
deletes the actor whose capability is *actorCap.
This type of call is used by the akill command. However, the call takes an
actor capability as argument instead of a local identi er used by the command.
The 0 value is returned if the c actor was successfully killed. Otherwise, a value
of -1 is returned, and errno is set to indicate the error:
value of errno interpretation
EFAULT cactorCap points to an illegal address
EINVAL value pointed to by cactorCap is not correct
EPERM the calling c actor is not trusted or its uid does not match
the uid of the target actor
It is interesting to note that if the capability corresponds to an actor which is
not currently a c actor, the actor is rst promoted to a c actor and then it is
132 Chapter 6. Actors
killed. We can check this using the myAkill application whose source is given
below:
--> cat $CHORUS/sources/actors/myAkill/myAkill.c
#include "utilities.h"
main(int argc, char *argv[ ]) {
KnCap actorCap;
int result;
readCap(argv + 1, &actorCap);
result = akill(&actorCap);
if(result == -1)
perror("akill");
else
fprintf(stderr, "actor has been deleted\n");
}

We then create an empty user actor with the actorCreate application we wrote
in 6.4.5:
--> neon actorCreate_u u a
started aid = 24
New actor's capability: 200000b4 869da80a 17 488
-->
As already stated, this new actor is not recognized as a c actor and cannot be
killed with the standard akill command:
--> rsh neon aps
UID AID NAME DBG GROUP
0 21 C_INIT 0 N/A
0 20 ADMIN 0 N/A
--> rsh neon arun cs | grep "no-name"
started aid = 24
200000b4 869da80a 00000017 00000488 0023 USER STARTED 000 <no-name>
--> rsh neon akill 23
C_INIT: akill: 23: cannot get actor capability (= it is not a c actor
-->
However, it can be killed with the myAkill application where its capability is
passed as argument:
--> neon myAkill_u 200000b4 869da80a 17 488
started aid = 24
actor has been deleted
--> rsh neon arun cs | grep "no-name"
started aid = 24
-->
6.6. Synchronizing c actors' threads' executions 133
6.6 Synchronizing c actors' threads' executions
By a mechanism similar to the wait primitive of Posix systems allowing a pro-
cess to wait for termination of a child process, a thread of a c actor may be
blocked until a given c actor terminates or stops due to tracing requested by
another actor through the atrace primitive. This primitive allows a c actor
to control the execution of another one and is essentially used for implement-
ing breakpoint debugging. Nevertheless, unlike Posix systems where processes
may be zombie, no such state exits for actors. Once it is terminated, an actor
completely disappears: thus, it will be impossible for a c actor to retrieve infor-
mation about termination of an actor if it calls one of the primitives we present
here after the target actor has exited. The function call
#include <am/await.h>
int awaits(
KnCap *cactorCap,
int *statusp
);

blocks the calling thread until the c actor whose capability is pointed to by
cactorCap terminates or stops because of tracing. When returning, if statusp is
not NULL, *statusp indicates why the c actor stopped or terminated (if the actor
terminated, the returned status corresponds to its exit value). If successful, the
function returns the local identi er of the c actor which terminated or stopped.
Otherwise -1 is returned and errno is set to one of the following values:
value of errno interpretation
EFAULT cactorCap or statusp point to an illegal address
EINVAL cactorCap corresponds to the calling actor
ESRCH cactorCap does not correspond to a valid c actor
EINTR the calling thread has been aborted
The await primitive only has a KnCap * parameter. Thus, its prototype is
#include <am/await.h>
int await(KnCap *cactorCap);

and a call is identical to awaits(cactorCap, NULL).


A set of macros may be used for interpreting the value of the status:
IS EXIT EVT(int status) returns 1 if the status indicates a termination
due to a call to exit;
134 Chapter 6. Actors
GET EXIT EVT(int status) returns the exit status of the c actor if it
terminated due to a call to exit (the value of
status is the value that was passed to exit);
IS KILL EVT(int status) returns 1 if the c actor terminated due to an
asynchronous event (exception, actorDelete,
akill);
GET KILL EVT(int status) if the c actor terminated due to an event,
it returns the corresponding event:
EVT ILL: illegal instruction exception;
EVT BKPT: breakpoint exception;
EVT FPE: oating point exception;
EVT SEGV: memory exception;
EVT AKILL: actor killed by akill;
EVT DELETE: actor killed by actorDelete;
IS STP EVT(int status) returns 1 if the status indicates that the
c actor was being debugged and caught
an exception;
GET STP EVT(int status) returns the event which caused the c actor
to stop.

In the following example, we rst develop a small application which may ter-
minate for di erent reasons depending on the value of its parameter.
The actor which is created
- kills itself through akill if the value is 0;
- deletes itself through actorDelete if the value is 1;
- provokes an exception by writing to a wrong address if the value is 2;
- terminates by calling exit otherwise and the corresponding value will be
passed as an argument to the call.
--> cat $CHORUS/sources/actors/await/awaited/waitedActor.c
#include "utilities.h"
#include <am/await.h>
int ind, status, arg1;
int *p = (int *) 0x47477000; /* an illegal address */
KnCap actorCap;

main(int argc, char *argv[ ]) {


actorSelf(&actorCap);
fprintCap(stderr, "Capability", &actorCap);
fprintf(stderr, "local identifier: %d\n", agetId( ));
6.6. Synchronizing c actors' threads' executions 135
switch(arg1 = atoi(argv[1])) {
case 0 : akill(K_MYACTOR); break;
case 1 : actorDelete(K_MYACTOR); break;
case 2 : *p = 0; break;
default : exit(arg1);
}
}

The second component of the example corresponds to an actor which creates


a c actor executing the previous command. It calls the afexecl primitive and
transmits its own parameters as parameter of this command. The actor then
waits for the termination of the new actor through a call to awaits. When
termination occurs, the actor is awoken and displays the information it got
when returning from the call:
--> cat $CHORUS/sources/actors/await/waiting/waitingActor.c
#include "utilities.h"
#include <am/afexec.h>
#include <am/await.h>
AcParam params;
KnCap actorCap;
char buf[256];
int status, result, event;

main(int argc, char *argv[ ]) {


if(argc < 2) {
fprintf(stderr, "bad usage: at least one argument\n");
exit(2);
}
fprintf(stderr, "********************************\n");
params.acSite = 0;
params.acFlags = AFX_USER_SPACE;
result = afexecl("/mnt/waitedActor_u",
&actorCap, &params, "waitedActor_u",
argv[1], NULL);
if(result == -1) {
perror("afexeclp");
exit(2);
}
fprintf(stderr, "new c_actor has been created .......\n");
fprintf(stderr, "********************************\n");
result = awaits(&actorCap, &status);
fprintf(stderr, "********************************\n");
if (result == -1) {
perror("awaits");
exit(2);
}
136 Chapter 6. Actors
fprintf(stderr, "actor with local identifier %d is terminated\n",
result);
if(IS_EXIT_EVT(status)) {
fprintf(stderr, " ==> exit(%d)\n", GET_EXIT_EVT(status));
exit(0);
}
if(IS_KILL_EVT(status)) {
fprintf(stderr, " ==> exception: ");
event = GET_KILL_EVT(status);
switch(event) {
case EVT_ILL : fprintf(stderr, "illegal instruction\n");
exit(0);
case EVT_BKPT : fprintf(stderr, "breakpoint\n");
exit(0);
case EVT_SEGV : fprintf(stderr, "memory exception\n");
exit(0);
case EVT_AKILL : fprintf(stderr, "actor killed\n");
exit(0);
case EVT_DELETE : fprintf(stderr, "actor deleted\n");
exit(0);
default : fprintf(stderr, "unknown exception\n");
exit(0);
}
}
if(IS_STP_EVT(status)) {
event = GET_STP_EVT(status);
fprintf(stderr, " ==> stopped: %d", event);
exit(0);
}
fprintf(stderr, " ==> unknown reason\n");
}

In the rst execution, the new c actor terminates with akill (0 is passed as an
argument):
--> neon waitingActor_u 0
started aid = 24
********************************
new c_actor has been created .......
********************************
Capability: 200000bd 869da80a 17 0
local identifier: 23
********************************
actor with local identifier 23 is terminated
==> exception: actor killed
6.6. Synchronizing c actors' threads' executions 137
In the second execution, the new c actor terminates with actorDelete (1 is
passed as an argument):
--> neon waitingActor_u 1
started aid = 24
********************************
new c_actor has been created .......
********************************
Capability: 200000bf 869da80a 17 4e4
local identifier: 23
********************************
actor with local identifier 23 is terminated
==> exception: actor deleted

In the third execution, the new c actor terminates with an exception (2 is


passed as an argument). We can observe that the exception message displays
the address where the memory fault occurred (0x47477000):
--> neon waitingActor_u 2
started aid = 24
********************************
new c_actor has been created .......
********************************
Capability: 200000c1 869da80a 17 1
local identifier: 23
Segmentation fault thread 8 PC ffff60a0 faultAddr 47477000
********************************
actor with local identifier 23 is terminated
==> exception: memory exception

The next execution provokes a termination through exit with the value 100
which is passed as argument:
--> neon waitingActor_u 100
started aid = 24
********************************
new c_actor has been created .......
********************************
Capability: 200000c3 869da80a 17 517
local identifier: 23
********************************
actor with local identifier 23 is terminated
==> exit(100)
-->

A nal execution provokes a termination by exit with the value 1000: we can
observe that the waiting actor gets this value as 1000 % 256:
138 Chapter 6. Actors
--> neon waitingActor_u 1000
started aid = 24
********************************
new c_actor has been created .......
********************************
Capability: 200000c5 869da80a 17 530
local identifier: 23
********************************
actor with local identifier 23 is terminated
==> exit(232)
-->
Chapter 7
Basic memory management
7.1 General concepts
7.1.1 The memory models and the features
Three di erent memory management models may be selected:
 the MEM FLAT model: this is the model implemented if no speci c feature is
enabled. Within this model, one unique unprotected supervisor address space
is de ned and shared by the kernel and all applications. Addresses directly
match the physical address space, the kernel simply provides services for allo-
cating memory. Applications may not allocate more memory than is physically
available;
 the MEM PROTECTED model: this provides the bene t of using the ex-
ibility and protection supported by the hardware (MMU). In particular, the
address space is divided into a supervisor space that is shared by the kernel
and all the supervisor actors, and a user address space that is used as a pri-
vate address space by every user actor. Nevertheless, no swap, no demand
paging and no mapper interface are provided. Thus, no more than physi-
cally available memory may be allocated. This model is implemented when
the VIRTUAL ADDRESS SPACE feature is enabled;
 the MEM VIRTUAL model: within this model corresponding to a system
where both VIRTUAL ADDRESS SPACE and ON DEMAND PAGING features are ac-
tive, a full service for virtual memory with swapping is provided. The following
services are provided:
- like the MEM PROTECTED model, multiple protected address spaces
for user actors;
- if secondary storage is available, the possibility for applications to allo-
cate more (virtual) memory than the available physical one. Pages are
automatically swapped in and out when it is appropriate;
139
140 Chapter 7. Basic memory management
- it exports two speci c abstractions: segments which are external rep-
resentations of data objects (like les or swap areas) managed by spe-
ci c external actors called mappers and local caches which are their
representation in the physical memory. A protocol is de ned by the
ON DEMAND PAGING feature which allows communication between the ker-
nel and mappers: this protocol is based on the IPC feature and de nes
the di erent kinds of messages that can be exchanged by the kernel and
the mappers.
The choice of a model has to be made depending on the hardware and the
software requirements (paging, protections between applications, . . . ) and in
this chapter we will focus on the MEM PROTECTED model and touch on
MEM FLAT. Chapter 15 is fully dedicated to the MEM VIRTUAL memory
model.
The memVersion application whose code is now given displays a message in-
dicating the type of memory management module which is being used by the
system:
--> cat $CHORUS/sources/memory/version/memVersion.c
#include <stdio.h>
#include <chorus.h>
#include <exec/chModules.h>
#include <mem/chMemConst.h>
int result, value;
main( ) {
result = sysGetConf(K_MODULE_MEM_NAME, K_GETCONF_VERSION, &value);
if (result != K_OK) {
fprintf(stderr, "error on sysGetConf: %s\n", strSysError(result));
exit(1);
}
switch(value) {
case K_MEM_VERSION_FLM:
printf("Flat memory\n");
break;
case K_MEM_VERSION_PRM:
printf("Protected memory\n");
break;
case K_MEM_VERSION_VM:
printf("Virtual memory\n");
}
}
-->
In the examples in this chapter, we will use two systems: carbon which uses
MEM FLAT and neon which uses MEM PROTECTED:
7.1. General concepts 141
--> carbon memVersion_u
started aid = 2
Flat memory
--> neon memVersion_u
started aid = 2
Protected memory
-->

7.1.2 Memory management


7.1.2.1 Supervisor and user address spaces
In current implementations for 32 bits processors, the address space of a proces-
sor is usually divided into two subsets and splits between both address spaces:
 the supervisor address space: contains the kernel data and code. It is also
used by the actors created with the supervisor privilege. This space is shared
by all actors but it can only be accessed by threads running with the supervisor
privilege level. This means that a same virtual address will be associated to
the same physical address in all actors where this address can be interpreted.
This level is usually characterized by a particular hardware state of the pro-
cessor which allows execution of privileged instructions (enabling interrupts,
input/output, . . . );
 the user address space: each user actor has its own user address de ned in
the corresponding range of addresses. This means that a given virtual address
has di erent interpretations in each actor. These interpretations generally lead
to di erent physical addresses except if the address points to a location within
a shared memory region.
The de nition of the ranges of addresses for both address spaces is de ned by
certain standards.
On some processors (PPC 603 for instance), these two address spaces do overlap
completely: the same logical address may match an address in the user address
space or in the supervisor address space depending on the execution context.
7.1.2.2 The page size
The physical memory is divided into pages: a page is the minimum allocatable
memory block. The page size can be retrieved by calling the function
int vmPageSize(void);

When the VIRTUAL ADDRESS SPACE feature is used, the size of a virtual page is
the same as the size of a physical page as returned by the previous function.
142 Chapter 7. Basic memory management
7.1.2.3 Parameters for memory management
The KnVmPar structure describes the memory management parameters used on
the running kernel. It has the following elds:
 usrStartAddr and usrEndAddr whose type is VmAddr: they de ne the bounds
of the user address space;
 svStartAddr and svEndAddr whose type is VmAddr: they de ne the bounds
of the supervisor address space;
 various other elds are meaningful within the MEM VIRTUAL model and
will be presented in chapter 15 along with the vmSetPar primitive which is used
to modify their values.
The KnVmStat structure is used to describe the state of the memory. It has the
following elds:
 par whose type is KnVmPar: it de nes the parameters of the memory manage-
ment as described above;
 hostMem, freeMem and lockMem whose type is VmSize: they describe the size
of the site's physical memory, the amount of currently non-allocated physical
memory, and the amount of currently allocated and non-swappable physical
memory.
This information may be retrieved by calling
#include <mem/chMem.h>
int vmStat(KnVmStat *stat);

The memoryStat application whose code is given below gets and displays the
parameters managed by the memory management module.
--> cat $CHORUS/sources/memory/memoryStat/memoryStat.c
#include <chorus.h>
KnVmStat stat;
main( ) {
int result;
printf("Page size: 0x%x [%d]\n", vmPageSize( ), vmPageSize( ));
result = vmStat(&stat);
printf("Total memory size: 0x%x [%d]\n", stat.hostMem, stat.hostMem);
printf("Amount of free memory: 0x%x [%d]\n", stat.freeMem, stat.freeMem);
printf("Amount of locked memory: 0x%x [%d]\n", stat.lockMem, stat.lockMem);
printf("Beginning of user address space: %p\n", stat.par.usrStartAddr);
printf("End of user address space: %p\n", stat.par.usrEndAddr);
printf("Beginning of superv. address space: %p\n", stat.par.svStartAddr);
printf("End of supervisor address space: %p\n", stat.par.svEndAddr);
}
-->
7.1. General concepts 143
 The application is rst executed on the carbon ix86 based machine where the
MEM FLAT memory management model is used:
--> carbon memoryStat_u
started aid = 2
Page size: 0x1000 [4096]
Total memory size: 0xfa0000 [16384000]
Amount of free memory: 0xd0f000 [13692928]
Amount of locked memory: 0x291000 [2691072]
Beginning of user address space: 0x0
End of user address space: 0xffffefff
Beginning of superv. address space: 0x0
End of supervisor address space: 0xffffefff
We can observe that there is a single address space: user address space and
supervisor address space are identical.
 We now execute the same application on the neon ix86 based machine within
the MEM PROTECTED memory management model:
--> neon memoryStat_u
started aid = 24
Page size: 0x1000 [4096]
Total memory size: 0xfa0000 [16384000]
Amount of free memory: 0xcae000 [13295616]
Amount of locked memory: 0x2f2000 [3088384]
Beginning of user address space: 0x8000000
End of user address space: 0xffffefff
Beginning of superv. address space: 0x0
End of supervisor address space: 0x7fffffff

The supervisor address space and the user address space correspond to di erent
segments of addresses. The division of the global address space between these
two address spaces is illustrated in the following diagram:
0xfffff000 one page out of address space
0xffffefff
user address space:
- each actor has its own space
- addresses are interpreted in
the context of the actor
0x80000000
0x7fffffff
supervisor address space:
it is shared by all
supervisor actors
and the system actors
0x0
144 Chapter 7. Basic memory management
7.2 Organization of an actor's address space
7.2.1 The concept of region
As we have seen when we displayed characteristics of actors with the cs com-
mand, the address space of an actor is divided in non-overlapping regions.
A region consists of a contiguous range of logical addresses having common at-
tributes. The number and type of these attributes depend on the memory man-
agement model being used, but they may be related to protection, inheritance
and paging. Due to the organization by pages a region consists of contiguous
logical pages. The starting address of a region is page-aligned and its size is
always a multiple of the page size.
As we will see in later sections, regions may be created and freed dynamically
by threads and may be shared by di erent actors.
We will use the following variant of the noDelay application in various examples
in this chapter (the de nition of a static array has just been added):
--> cat $CHORUS/sources/utilities/noDelay3/noDelay3.c
#include <chorus.h>
char t[0x8000]; /* an array of 8 pages in static data region */
main( ) {
threadDelay(K_NOTIMEOUT);
}

7.2.2 Organization of user c actors' address space


The MEM VIRTUAL memory management model will be discussed in chapter
15.
7.2.2.1 The MEM FLAT model
We rst execute two samples of the user version of the noDelay3 application
on the carbon system where the MEM FLAT management memory model is
used (no memory management feature has been selected). We get the memory
organization of the corresponding c actors by calling the cs utility:
--> carbon-n noDelay3_u & (= create a rst user actor
[1] 4234
--> started aid = 2

--> carbon-n noDelay3_u & (= create a second user actor


[2] 4238
--> started aid = 22
7.2. Organization of an actor's address space 145
--> rsh carbon arun cs -la 2 (=
get attributes of the rst user c actor
started aid = 23
ChorusOS r4.0.0 Site 0 Time 30m 46
ACTOR-UI KEY LID TYPE STATUS TH# NAME
2000001c 869da806 00000002 00000000 0002 USER STARTED 001 noDelay3_u
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0009 140 00000000 00000000 121a0 0- 1- 0 main
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
2000001c 869da806 def no 17320 0 2000001c 869da80a
START SIZE OFFSET ALLOC OPTIONS
00081000 00003000 00000000 00003000 EX FZ AW
00093000 00009000 00000000 00009000 WR FZ AW
00084000 00004000 00000000 00004000 WR FZ AW
002bb000 00002000 00000000 00002000 WR AW
--> rsh carbon arun cs -la 22 (=
get attributes of the second user c actor
started aid = 23
ChorusOS r4.0.0 Site 0 Time 31m 14
ACTOR-UI KEY LID TYPE STATUS TH# NAME
2000001e 869da806 00000016 00000044 0022 USER STARTED 001 noDelay3_u
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0008 140 00000000 00000000 1232c 0- 1- 0 main
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
2000001e 869da806 def no 171e4 0 2000001e 869da80a
START SIZE OFFSET ALLOC OPTIONS
002bd000 00003000 00000000 00003000 EX FZ AW
002c0000 00009000 00000000 00009000 WR FZ AW
002c9000 00004000 00000000 00004000 WR FZ AW
002ce000 00002000 00000000 00002000 WR AW
-->

We can observe that the user c actors executing the same application have the
same number of regions, and corresponding regions in both actors have the
same attributes (size and options), but that the starting addresses are di erent
as addresses are physical addresses.
Each c actor has four regions when created:
- the rst region (3 pages) contains the program;
- the second region (9 pages) is the static data region;
- the third region (4 pages) is the stack;
- the fourth region (2 pages) contains the arguments of the main thread.
Each region has attributes: it may be writable (WR), executable (EX), initialized
to zero (FZ) or implemented anywhere (AW)). These di erent attributes will be
presented in 7.2.5.
146 Chapter 7. Basic memory management
7.2.2.2 The MEM PROTECTED model
We now execute two samples of the user version of the noDelay3 application on
the neon system using the MEM PROTECTED memory management model.
We then get the memory organization of the corresponding c actors by calling
the cs utility:
--> neon-n noDelay3_u & (= create a rst user actor
[1] 8879
--> started aid = 2
--> rsh neon arun cs -la 2 (= get its characteristics
started aid = 22
ChorusOS r4.0.0 Site 0 Time 1h 17m 33
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000022 869da80a 00000002 0000007b 0002 USER STARTED 001 noDelay3_u
............................
START SIZE OFFSET ALLOC OPTIONS
fffec000 00006000 00000000 00006000 WR
ffff3000 00009000 00000000 00009000 WR
ffffc000 00003000 00000000 00003000 EX
--> neon-n noDelay3_u & (= create a second user actor
[2] 8883
--> started aid = 22
--> rsh neon arun cs -la 22 (= get its characteristics
started aid = 23
ChorusOS r4.0.0 Site 0 Time 1h 20m 48
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000024 869da80a 00000016 00000000 0022 USER STARTED 001 noDelay3_u
............................
START SIZE OFFSET ALLOC OPTIONS
fffec000 00006000 00000000 00006000 WR
ffff3000 00009000 00000000 00009000 WR
ffffc000 00003000 00000000 00003000 EX
-->
We can observe from these results that the address space of user actors which
execute the same command contains exactly the same logical regions: same
starting address (the memory management unit will translate these logical ad-
dresses to the corresponding physical addresses), same size and same attributes.
When it is created by the actor manager, a user c actor has three regions: their
characteristics (start address, size and attributes) appear in the results. In
contrast to the results within the MEM FLAT model, two regions have been
merged into a single one. In both actors:
- the region starting at 0xffffc000 is an executable region of 3 pages which
contains the program;
7.2. Organization of an actor's address space 147
- the region starting at 0xfffec000 is a 6 page writable region. It rst con-
tains the stack of the main thread of the actor: its size is the value of the
kern.exec.dflUsrStackSize tunable parameter whose default value is 0x4000
(4 pages). The two other pages of this region contain a copy of the parameters
of the main function;
- the region starting at 0xffff3000 is a 9 page writable region which corre-
sponds to the static data region.
A system stack is also automatically allocated in the supervisor space (and thus,
does not appear in the address space of the user actor) when the main thread
is created. The size of this stack is de ned by the value of the con guration's
parameter kern.exec.dflSysStackSize whose default value is 0x3000.

7.2.3 Organization of supervisor c actors


We now create two supervisor actors on the same system for the application:
--> neon-n noDelay3_s.r &
[3] 8888
--> started aid = 23

--> rsh neon arun cs -la 23


started aid = 24
ChorusOS r4.0.0 Site 0 Time 1h 49m 27
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000027 869da80a 00000017 00000000 0023 SUP STARTED 001 noDelay3_s.r
............................
START SIZE OFFSET ALLOC OPTIONS
7bbe6000 00002000 00000000 00002000 WR SU
7bbe9000 00009000 00000000 00009000 WR SU
7bbf2000 00003000 00000000 00003000 EX SU
--> neon-n noDelay3_s.r &
[4] 8892
--> started aid = 24

--> rsh neon arun cs -la 24


started aid = 25
ChorusOS r4.0.0 Site 0 Time 1h 50m 45
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000029 869da80a 00000018 00000000 0024 SUP STARTED 001 noDelay3_s.r
............................
7bbd3000 00002000 00000000 00002000 WR SU
7bbda000 00009000 00000000 00009000 WR SU
7bbe3000 00003000 00000000 00003000 EX SU
-->
148 Chapter 7. Basic memory management
We can observe from these results that, if both supervisor actors within the same
program have the same number of regions (having the same characteristics in
both actors), unlike user actors, the corresponding regions do not have the same
starting address (although they are the same size): supervisor actors share one
unique address space (the supervisor address space). The situation looks quite
similar to the one encountered for user actors within the MEM FLAT memory
model. Nevertheless, it is di erent. Addresses that are displayed are logical
addresses and not physical addresses: the memory management unit translates
these logical addresses onto their corresponding physical ones.
The regions displayed for each actor are the regions created when the AM loader
loaded the executable le; these regions are freed when the actor is deleted.
All supervisor actors may access all regions in the supervisor address space; this
means that a supervisor actor could corrupt the data of another one.
Both supervisor c actors have 3 regions once created (the SU attribute indicates
the fact that only supervisor threads may access them):
- the executable regions starting at addresses 7bbf2000 and 7bbe3000 are their
program regions;
- the regions starting at addresses 7bbe9000 and 7bbda000 are their static data
areas;
- the regions starting at addresses 7bbe6000 and 7bbd3000 contain a copy of
the parameters.
The main thread of a supervisor c actor is a supervisor thread and has no user
stack: it uses the system stack which is automatically allocated when the c actor
is created.

7.2.4 Regions' descriptors


The kernel manages a list of memory regions of the various actors. From the
user point of view, ranges of addresses are described by region descriptors whose
type is KnRgnDesc. Its de nition is given below:
typedef struct {
VmAddr startAddr; /* starting address of the region */
VmSize size; /* size (number of bytes) of the region */
VmFlags options; /* control fields for attributes */
VmAddr endAddr; /* to define an interval when creating a region */
void* opaque1; /* reserved for advanced use */
VmFlags opaque2; /* reserved for advanced use */
} KnRgnDesc;
7.2. Organization of an actor's address space 149
This type of descriptor is used to de ne a range of addresses on which an opera-
tion (a system call) must be applied and also to get information when returning
from such a call.
7.2.5 Regions' attributes
The options eld of the KnRgnDesc type describes the attributes of a region
and is built as a combination (using bitwise operators) of di erent constants
corresponding to attributes. These attributes are recorded in the kernel's in-
ternal descriptor of the region. The kernel will merge a number of adjacent
regions with the same set of attributes in its list. A region is given a number
of attributes at creation time. Some of these attributes may be changed at any
time. It is important to note that modifying some attributes may lead to:
 splitting a kernel's region into two separate regions (if the modi cation does
not apply to the whole region, but just to an initial segment or a terminal seg-
ment) or into three separate regions (if the modi cation applies to a segment
in the middle of a region);
 coalesce two or three adjacent regions into a single one if the modi cation
gives the same nal attributes to these contiguous regions.
For every region's attribute there exists a speci c ag which may be used for
de ning the options eld of a KnRgnDesc object. They may be classi ed in
di erent categories listed below. For every attribute we will also give the ab-
breviation (if any) that is used by the cs command to display it.
7.2.5.1 Initialization attributes
These are special attributes which can be used to perform particular operations
or taking optional decisions when a region is created or when pages of the region
are loaded for the rst time. These attributes are set when the region is created,
for example when calling the rgnAllocate primitive.
 K FILLZERO [FZ]: the region is lled with zeroes (otherwise the contents of
memory are unspeci ed);
 K ANYWHERE [AW]: the choice of allocation address of the region is left to the
system;
 K RESTRICTIVE: indicates that the region may be allocated wherever possible
between two speci ed addresses (between startAddr and endAddr of a given
region's descriptor).
7.2.5.2 Protection attributes
They specify the operations that are allowed on the region:
 K READABLE: read access is allowed for the region. In the current version of
ChorusOS, this access is always provided for any region as it is de ned as 0;
150 Chapter 7. Basic memory management
 K WRITABLE [WR]: the region may be modi ed (otherwise it is only readable);
 K EXECUTABLE [EX]: execution access is allowed;
 K SUPERVISOR [SU]: access is allowed only for threads executing with the SU-
PERVISOR privilege.
The way these di erent ags are implemented may vary depending on the un-
derlying hardware: for example, some hardware always grants read access, some
do not have explicit execution protection mechanisms.
Within the three memory management models, it is possible to change the value
of these attributes by calling the primitive
#include <mem/chMem.h>
int rgnSetProtect(
KnCap *targetActorCap,
KnRgnDesc *rgnDesc
);

The e ects of such a call apply to the entire range of addresses speci ed by the
startAddr and size elds of the object pointed to by rgnDesc within the actor
named by targetActorCap. The starting address startAddr must be page-aligned
and if the ending address (startAddr + size - 1) is within a page it is rounded
to the end of that page.
The e ects of the call are the following:
 the K READABLE, K EXECUTABLE, K WRITABLE or K SUPERVISOR attributes in
the kernel's descriptor are replaced by their correspondent in the options eld
of the region's descriptor pointed to by rgnDesc;
 protections are modi ed at the low level (page table entries).
The function returns K OK upon success and a negative error code otherwise.
return value error type
K EINVAL inconsistent actor capability
K EUNKNOWN unknown actor
K EFAULT address out of address space
K EROUND starting address not page-aligned
The modAttr application whose code is given below can be used to modify the
protection attributes of the regions of an actor in a given range of addresses by
calling rgnSetProtect. The arguments of the command are the capability of
the target actor, the starting address, the size of the range of addresses to be
considered, and the new value of the protection attribute (w, x or r):
7.2. Organization of an actor's address space 151
--> cat $CHORUS/sources/memory/modAttr/modAttr.c
#include "utilities.h"
main(int argc, char*argv[ ]) {
KnRgnDesc rgnDesc;
int result;
KnCap targetActorCap;
readCap(argv + 1, &targetActorCap);
sscanf(argv[5], "%x", &rgnDesc.startAddr);
sscanf(argv[6], "%x", &rgnDesc.size);
switch (argv[7][0]) {
case 'w' : rgnDesc.options = K_WRITABLE; break;
case 'x' : rgnDesc.options = K_EXECUTABLE; break;
default : rgnDesc.options = K_READABLE;
}
result = rgnSetProtect(&targetActorCap, &rgnDesc);
if (result != K_OK)
fprintf(stderr, "error on rgnSetProtect: %s\n", strSysError(result));
}
-->

7.2.5.3 Virtual memory speci c attributes


Various additional attributes are provided with the MEM VIRTUAL model,
that is, when the ON DEMAND PAGING feature is active, they will be presented in
chapter 15.
7.2.6 Virtual addresses and physical addresses
If the MEM FLAT memory model is used, virtual addresses match physical
addresses directly. On the other hand, if another model is used (that is as
soon as the VIRTUAL ADDRESS SPACE feature is active), this is not the case;
applications manipulate logical (or virtual) addresses which are translated by
the memory management module onto physical addresses. The API of the
VIRTUAL ADDRESS SPACE feature exports a primitive to retrieve the physical
address corresponding to a given logical address:
#include <mem/chMem.h>
int vmPhysAddr(
KnCap *actorCap,
VmAddr virtAddr,
PhAddr *physAddr
);
Given a virtual address virtAddr in the address space of an actor whose capa-
bility is pointed to by actorCap, a call to vmPhysAddr returns in *physAddr the
physical address mapped at virtAddr in the actor's address space.
152 Chapter 7. Basic memory management
If actorCap is K SVACTOR, the supervisor address space is used. The virtual
address must have been mapped into the physical memory when this type of
call takes place.
The call returns the value K OK if it is successful and a negative value (K EFAULT,
K EINVAL or K EUNKNOWN) otherwise.
We give below the source of a command for getting the physical address corre-
sponding to a logical one:
 if the command has only one parameter, the command assumes it corresponds
to an address in the supervisor address space;
 otherwise the command interprets the rst four parameters as a capability
and the fth one as a virtual address.
--> cat $CHORUS/sources/memory/physAddr/physAddr.c
#include "utilities.h"
KnCap targetActorCap;
VmAddr virtAddr;
PhAddr physAddr;
int result;
main(int argc, char *argv[ ]) {
if(argc == 2) {
sscanf(argv[1], "%x", &virtAddr);
result = vmPhysAddr(K_SVACTOR, virtAddr, &physAddr);
}
else {
readCap(argv + 1, &targetActorCap);
sscanf(argv[5], "%x", &virtAddr);
result = vmPhysAddr(&targetActorCap, virtAddr, &physAddr);
}
if(result < 0) {
fprintf(stderr, "error on vmPhysAddr: %s\n", strSysError(result));
exit(1);
}
fprintf(stderr, "virt addr %x ==> phys addr %p", virtAddr, physAddr);
if (argc == 2)
fprintf(stderr, " in supervisor space\n");
else
fprintCap(stderr, " in", &targetActorCap);
}
We rst use this application to get the physical addresses corresponding to
logical addresses in the private address space of the user actors we created on
the neon system in 7.2.2.2 for the noDelay3 application (local identi ers 2 and
22). We can check that the same virtual address in the address spaces of two
actors corresponds to di erent physical addresses, which ensures the privacy of
the data:
7.2. Organization of an actor's address space 153
--> neon physAddr_u 20000022 869da80a 2 7b 0xffff3000
started aid = 25
virt addr ffff3000 ==> phys addr 0xe9c000 in: 20000022 869da80a 2 7b
--> neon physAddr_u 20000024 869da80a 16 0 0xffff3000
started aid = 25
virt addr ffff3000 ==> phys addr 0xea4000 in: 20000024 869da80a 16 0
--> neon physAddr_u 20000022 869da80a 2 7b 0xffffc030
started aid = 25
virt addr ffffc030 ==> phys addr 0xea3030 in: 20000022 869da80a 2 7b
--> neon physAddr_u 20000024 869da80a 16 0 0xffffc030
started aid = 25
virt addr ffffc030 ==> phys addr 0xec9030 in: 20000024 869da80a 16 0
-->

We now do the same for the C INIT supervisor actor:


--> rsh neon aps
UID AID NAME DBG GROUP
0 24 noDelay3_s.r 0 N/A
0 23 noDelay3_s.r 0 N/A
0 22 noDelay3_u 0 N/A
0 2 noDelay3_u 0 N/A
0 21 C_INIT 0 N/A
0 20 ADMIN 0 N/A
--> rsh neon arun cs -la 21
started aid = 25
ChorusOS r4.0.0 Site 0 Time 3h 0m 51
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000015 869da80a 00000015 00000001 0021 SUP STARTED 001 C_INIT
......................
START SIZE OFFSET ALLOC OPTIONS
400e4000 00017000 00000000 00017000 EX SU
400fb000 00002000 00000000 00002000 WR SU
7bbd1000 00002000 00000000 00002000 WR SU
7bbff000 00002000 00000000 00002000 WR SU
--> neon physAddr_u 20000015 869da80a 15 1 0x400e5555
started aid = 25
virt addr 400e5555 ==> phys addr 0x230555 in: 20000015 869da80a 15 1
-->

We nally do the same without specifying the capability of the actor:


--> neon physAddr_u 0x400e5555
started aid = 25
error on vmPhysAddr: Bad address (=
need to be supervisor actor
--> neon physAddr_s.r 0x400e5555
started aid = 25
virt addr 400e5555 ==> phys addr 0x230555 in supervisor space
-->
154 Chapter 7. Basic memory management
The next gure summarizes the correspondence (mapping) between logical ad-
dresses and physical addresses (for addresses in either the user address space of
user actors or in the supervisor address space):
user ffffc000 ecb000
actor
2 ffff3000
ebc000

user ffffc000
actor e9d000

22 ffff3000

e82000

400e5000
246555

physical
physical memory addresses
supervisor address
space

7.2.7 Displaying all regions of an actor


7.2.7.1 The rgnStat function
It is possible to retrieve in an application the regions of an actor in a range of
addresses by calling the primitive
#include <mem/chMem.h>
int rgnStat(
KnCap *targetActorCap,
KnRgnDesc *rgnDesc,
KnRgnStat *stat,
unsigned statSize
);
The call returns the total number of regions which are attached to the actor
whose capability is de ned by targetActorCap and which are in the range of ad-
dresses [rgnDesc.startAddr : rgnDesc.size[. If targetActorCap is K SVACTOR,
the call takes all the regions of the supervisor address range into account. In-
formation about these di erent regions (starting address, size and attributes)
is written into memory at stat address as objects of the type KnRgnStat. The
statSize argument de nes the size of the allocated space at that address (it is
the size as a number of bytes and not as a number of items). In any case, only
statSize bytes will be written to memory, at most. Hence information on some
regions may not be written at all if the allocated space is not sucient.
7.2. Organization of an actor's address space 155
The KnRgnStat structure is de ned as
typedef struct {
VmAddr startAddr;
VmSize size;
VmFlags options;
void* opaque1;
VmFlags opaque2;
VmOffset offset;
KnLcStat lcstat;
} KnRgnStat;

The last four elds are only meaningful within the MEM VIRTUAL memory
model.
7.2.7.2 Examples
The doRegionStat function (whose code is given below) loads within the statAr-
ray bu er the characteristics of the rst numEntries regions of the complete
address space of an actor whose capability is pointed to by actorCap. Address
space is scanned between 0 and the K MAXVMSIZE symbolic value which de nes
the greatest legal virtual address (in our examples it is 0xffffefff).
--> cat $CHORUS/sources/memory/regions/regions.c
#include <stdio.h>
#include <chorus.h>

int doRegionStat(KnCap *actorCap, KnRgnStat *statArray, int numEntries) {


int result;
static KnRgnDesc rgnDesc;
rgnDesc.startAddr = 0x0;
rgnDesc.size = K_MAXVMSIZE;
result = rgnStat(actorCap, &rgnDesc, statArray,
numEntries * sizeof(KnRgnStat));
if(result < 0)
fprintf(stderr, "error on rgnStat: %s\n", strSysError(result));
return result;
}

The printRegion function displays the elds of a KnRgnStat entry: it also takes
into account the di erent attributes speci c to the MEM VIRTUAL module
and prints the values of the opaque elds which are meaningful only with that
model:
#include <stdio.h>
#include <chorus.h>
#include <exec/chModules.h>
#include <mem/chMemConst.h>
int doRegionStat(KnCap *actorCap, KnRgnStat *statArray, int numEntries);
156 Chapter 7. Basic memory management
void printRegion(KnRgnStat *rgnStat) {
int result, value;
result = sysGetConf(K_MODULE_MEM_NAME, K_GETCONF_VERSION, &value);
printf("start = %-10p size = 0x%-6x [%p]",
rgnStat -> startAddr, rgnStat -> size, rgnStat -> options);
if(rgnStat -> options & K_ANYWHERE) printf("AW ");
if(rgnStat -> options & K_WRITABLE) printf("WR ");
if(rgnStat -> options & K_EXECUTABLE) printf("EX ");
if(rgnStat -> options & K_SUPERVISOR) printf("SU ");
if(rgnStat -> options & K_INHERITSHARE) printf("IS ");
if(rgnStat -> options & K_INHERITCOPY) printf("IC ");
if(rgnStat -> options & K_NODEMAND) printf("ND ");
if(rgnStat -> options & K_NOWAITFORMEMORY) printf("NW ");
if(rgnStat -> options & K_FILLZERO) printf("FZ ");
if(rgnStat -> options & K_NOSWAPOUT) printf("NS ");
if (value == K_MEM_VERSION_VM)
printf(" [%x] [%x]\n", rgnStat -> opaque1, rgnStat -> opaque2);
else putchar('\n');
}

Finally, the listRegions application uses the previous functions to display the
characteristics of the regions of an actor whose capability is transmitted as
argument to the command:
--> cat $CHORUS/sources/memory/regions/listRegions.c
#include "utilities.h"
#define REGIONSMAXNUMBER 30
KnRgnStat statArray[REGIONSMAXNUMBER];
main(int argc, char *argv[ ]) {
int ind, result;
KnCap actorCap;
readCap(argv + 1, &actorCap);
result = doRegionStat(&actorCap, statArray, REGIONSMAXNUMBER);
if(result >= 0) {
printf("The actor has %d regions\n", result);
if (result > REGIONSMAXNUMBER) result = REGIONSMAXNUMBER;
for(ind = 0; ind < result; ind ++) printRegion(statArray + ind);
}
}

We use this application to display the di erent regions of two c actors previously
created and loaded on the neon system (user actor 2 and supervisor actor 24):
--> neon listRegions_u 20000022 869da80a 2 7b (= user actor 2
started aid = 25
The actor has 3 regions
start = 0xfffec000 size = 0x6000 [0x2]WR
start = 0xffff3000 size = 0x9000 [0x2]WR
start = 0xffffc000 size = 0x3000 [0x1]EX
7.3. The Posix interface 157
--> neon listRegions_u 20000029 869da80a 18 0 (= supervisor actor 24
started aid = 25
The actor has 3 regions
start = 0x7bbd3000 size = 0x2000 [0x6]WR SU
start = 0x7bbda000 size = 0x9000 [0x6]WR SU
start = 0x7bbe3000 size = 0x3000 [0x5]EX SU
-->

7.3 The Posix interface


A c actor may use the malloc (and its derivate calloc or realloc) and free
Posix functions to allocate or free memory dynamically. Here is a simple ex-
ample which uses this interface to allocate space in the calling actor and the
doRegionStat and printRegion functions to display the evolution of regions
in the actor:
--> cat $CHORUS/sources/memory/malloc/regionsMalloc.c
#include <stdio.h>
#include <stdlib.h>
#include <chorus.h>
int externIntVar;
char *p_malloc;
KnRgnStat statArray[10];

main( ) {
int stackIntVar,
result,
ind;
printf("[program] address of stackIntVar: %p\n", &stackIntVar);
printf("[program] address of externIntVar: %p\n", &externIntVar);
result = doRegionStat(K_MYACTOR, statArray, 10);
printf("Actor has %d regions\n", result);
for(ind = 0; ind < result; ind ++) {
printf("[%d] ==> ", ind);
printRegion(statArray + ind);
}
p_malloc = (char *) malloc(0x9000);
printf("\nAfter malloc .....\n");
printf("[program] malloc returns %p\n", p_malloc);
result = doRegionStat(K_MYACTOR, statArray, 10);
printf("Actor has now %d regions\n", result);
for(ind = 0; ind < result; ind ++) {
printf("[%d] ==> ", ind);
printRegion(statArray + ind);
}
}
158 Chapter 7. Basic memory management
--> neon regionsMalloc_u
started aid = 25
[program] address of stackIntVar: 0xffff3fcc
[program] address of externIntVar: 0xffff58a0
Actor has 3 regions
[0] ==> start = 0xfffee000 size = 0x6000 [0x2]WR
[1] ==> start = 0xffff5000 size = 0x1000 [0x2]WR
[2] ==> start = 0xffff6000 size = 0x9000 [0x1]EX
After malloc .....
[program] malloc returns 0xfffe4008
Actor has now 3 regions
[0] ==> start = 0xfffe4000 size = 0x10000 [0x2]WR
[1] ==> start = 0xffff5000 size = 0x1000 [0x2]WR
[2] ==> start = 0xffff6000 size = 0x9000 [0x1]EX
-->

We can observe from these results that:


 before executing the malloc operation, the actor has 3 regions as would
any actor created and dynamically loaded by the AM on a system within the
MEM PROTECTED memory management model. The stackIntVar variable
is allocated on the stack of the main thread: its address (0xffff3fcc) belongs
to the range of addresses associated with the region marked by [0] in the rst
list of regions. The externIntVar variable is an external static variable and
its address (0xffff58a0) belongs to the range of addresses associated with the
region marked by [1];
 once malloc has been executed, the actor still has 3 regions but the region
marked by [0] has just been extended. The malloc call returns the value
0xfffea008 inside that region.

7.4 Creating a new region


7.4.1 The rgnAllocate primitive
Within the three memory management models, a simple service is provided to
create a region in a given actor by calling the primitive
#include <mem/chMem.h>
int rgnAllocate(
KnCap *actorCap,
KnRgnDesc *rgnDesc
);

In this type of a call, actorCap points to the capability of the target actor (the
K MYACTOR value corresponds to the current actor). Unless the region is later
7.4. Creating a new region 159
shared with other actors or explicitly freed, it will be destroyed when the actor
is deleted.
The K SVACTOR value may be used: in that case the region will be allocated in
the supervisor address space and will not be attached to any actor. It will have
to be explicitly deallocated.
The rgnDesc parameter points to a descriptor containing the characteristics of
the new region:
 unless K ANYWHERE or K RESTRICTIVE is set in the options eld of the object
pointed to by rgnDesc, the startAddr eld of this object is the starting address
of the new region and which must be page-aligned and whose size eld de nes
the size of the new region;
 if K ANYWHERE is set, the kernel selects the starting address and, upon return
from the function, the value of the startAddr eld of the object pointed to by
rgnDesc is the new region's starting address;
 if K RESTRICTIVE is set, the region is allocated anywhere in the range de ned
by the startAddr and endAddr elds of the object pointed to by rgnDesc;
 other ags may be used for selecting speci c attributes of the new region
(K FILLZERO, K NODEMAND, . . . ).
Certain particular properties should be noted:
 the new region may not overlap with an existing one;
 if the K SUPERVISOR ag is set, only supervisor threads will be allowed to
access the new region.
A successful call returns K OK, otherwise a negative value to indicate that an
error occurred:
return value error type
K EUNKNOWN unknown actor
K EINVAL inconsistent actor capability or ags
K EFAULT address out of address space
K EROUND starting address not page-aligned
K ESPACE rgnDesc.size is 0 or invalid value of rgnDesc.startAddr
K EOVERLAP allocation of the region would lead to overlapping
K ENOMEM system out of resources

7.4.2 Examples
7.4.2.1 Internal allocation
We rst examine di erent situations when trying to allocate regions inside the
calling actor through the internalAlloc u and internalAlloc s.r binaries
corresponding to the following application:
160 Chapter 7. Basic memory management
--> cat $CHORUS/sources/memory/internalAlloc/internalAlloc.c
#include <stdio.h>
#include <chorus.h>
int *addr, result;
KnRgnDesc rgnDesc;

main( ) {
KnCap actorCap;
/* trying to allocate a region in the supervisor address space */
rgnDesc.startAddr = 0x66666300;
rgnDesc.size = 0x1000;
rgnDesc.options = K_WRITABLE;
result = rgnAllocate(K_MYACTOR, &rgnDesc);
if (result < 0)
fprintf(stderr, "error on first rgnAllocate: %s\n\n",
strSysError(result));
else
fprintf(stderr, "first rgnAllocate succeeds\n\n");
/* trying to allocate a region in the supervisor address space */
/* with K_SUPERVISOR attribute */
rgnDesc.startAddr = 0x66666000;
rgnDesc.options |= K_SUPERVISOR;
result = rgnAllocate(K_MYACTOR, &rgnDesc);
if (result < 0)
fprintf(stderr, "error on second rgnAllocate: %s\n\n",
strSysError(result));
else
fprintf(stderr, "second rgnAllocate succeeds\n\n");
/* allocating a region anywhere in the user address space */
/* the address specified in startAddr is not used */
rgnDesc.options = K_WRITABLE | K_ANYWHERE;
result = rgnAllocate(K_MYACTOR, &rgnDesc);
if (result < 0)
fprintf(stderr, "error on third rgnAllocate: %s\n",
strSysError(result));
else {
fprintf(stderr, "address of new region: %p\n", rgnDesc.startAddr);
addr = (int *) rgnDesc.startAddr;
*addr = 3333;
fprintf(stderr, "at address %p: %d\n\n", addr, *addr);
}
/* allocating a region with K_SUPERVISOR at address */
/* 0x66666000 in the supervisor address space */
rgnDesc.startAddr = 0x66666000;
rgnDesc.options = K_WRITABLE | K_SUPERVISOR;
result = rgnAllocate(K_MYACTOR, &rgnDesc);
7.4. Creating a new region 161
if (result < 0)
fprintf(stderr, "error on fourth rgnAllocate: %s\n\n",
strSysError(result));
else {
addr = (int *) 0x66666000;
*addr = 1200;
fprintf(stderr, "in 0x66666000: %d\n", *addr); }
/* allocating a region in the user address space */
rgnDesc.startAddr = 0x90000000;
rgnDesc.options = K_WRITABLE;
result = rgnAllocate(K_MYACTOR, &rgnDesc);
if (result < 0)
fprintf(stderr, "error on fifth rgnAllocate: %s\n\n",
strSysError(result));
else {
addr = (int *) 0x90000000;
*addr = 1200;
fprintf(stderr, "in 0x90000000: %d\n\n", *addr); }
/* allocating a region with K_SUPERVISOR at address */
/* 0x90004000 in the user address space */
rgnDesc.startAddr = 0x90004000;
rgnDesc.options = K_WRITABLE | K_SUPERVISOR;
result = rgnAllocate(K_MYACTOR, &rgnDesc);
if (result < 0)
fprintf(stderr, "error on sixth rgnAllocate: %s\n\n",
strSysError(result));
else {
addr = (int *) 0x90004000;
*addr = 1200;
fprintf(stderr, "in 0x90004000: %d\n", *addr); }
}
We rst execute the user version:
--> neon internalAlloc_u
started aid = 25
error on first rgnAllocate: Alignment error

error on second rgnAllocate: Address space violation

address of new region: 0xffff4000


at address 0xffff4000: 3333

error on fourth rgnAllocate: Address space violation

in 0x90000000: 1200

Segmentation fault thread 11 PC ffff62ca faultAddr 90004000


-->
162 Chapter 7. Basic memory management
These successive results show that:
 the starting address of a region must be page-aligned (a K EROUND error is
returned otherwise, as in the rst request);
 a region of the supervisor address space cannot be allocated by a user actor
(K ESPACE error) even when using the K SUPERVISOR ag;
 a region may be allocated either to a given address (if the new region does
not overlap an existing one) or within the K ANYWHERE ag. In that case the
starting address is returned in the region's descriptor (it is an address in the
user address space since the actor is a user actor) and this region may be used
according to its protection attributes (here a write and a read are performed);
 a region may be allocated in the user address space with the K SUPERVISOR
attributes. However, this region can only be accessed by actors' threads when
executing with supervisor privilege. Otherwise, an exception occurs.
We now execute the supervisor version of the command:
--> neon internalAlloc_s.r
started aid = 25
error on first rgnAllocate: Alignment error

second rgnAllocate succeeds

address of new region: 0x7bbd9000


at address 0x7bbd9000: 3333

error on fourth rgnAllocate: Region overlapping

error on fifth rgnAllocate: Address space violation

error on sixth rgnAllocate: Address space violation


-->
These results illustrate that:
 the region must be page-aligned;
 the request for a region at a given address in the supervisor address space
succeeds if the new region does not overlap an existing one. A region can be
safely allocated in the supervisor address space using the K ANYWHERE ag;
 a request for allocating a region in the user address space fails (K ESPACE
error).
7.4.2.2 External allocation
We now give the source of an application that can be used to allocate a region
in a given actor:
7.4. Creating a new region 163
 the rst four parameters de ne the capability of the actor;
 the fth parameter gives the hexadecimal starting address of the region, and
the sixth the size of the region in hexadecimal form;
 the other optional parameters give a list of attributes of the regions as integers:
an attribute is selected by giving its position in the attributes array as de ned
in the source (this list includes attributes of the MEM VIRTUAL module).
--> cat $CHORUS/sources/memory/allocRgn/allocRgn.c
#include "utilities.h"
KnRgnDesc rgnDesc;
int ind, result;
int attributes[ ] = { K_ANYWHERE, K_WRITABLE, K_EXECUTABLE, K_SUPERVISOR,
K_INHERITSHARE, K_INHERITCOPY, K_NODEMAND,
K_NOWAITFORMEMORY, K_FILLZERO, K_NOSWAPOUT };
KnCap targetActorCap;
int flag;
main(int argc, char *argv[ ]) {
if (argc < 8) {
fprintf(stderr, "Bad usage\n");
exit(1); }
readCap(argv + 1, &targetActorCap);
sscanf(argv[5], "%x", &rgnDesc.startAddr);
sscanf(argv[6], "%x", &rgnDesc.size);
for (ind = 7; ind < argc; ind ++) {
flag = atoi(argv[ind]);
if (flag >= 0 && flag < 10)
rgnDesc.options |= attributes[flag];
}
result = rgnAllocate(&targetActorCap, &rgnDesc);
if(result < 0)
fprintf(stderr, "error on rgnAllocate: %s\n", strSysError(result));
}

a) Regions in a user actor


We now use this application to create a region in the user c actor identi ed by
2 that we created in 7.2.2.2 on the neon system. The listRegions application
is also used to control the e ect of our requests.
In the rst request, the ag K ANYWHERE is set (it is the only attribute which is
used). The address that we transmit is not used (it was a wrong address since
it was not page aligned and belongs to the supervisor address space).
--> rsh neon arun cs -la 2
.....................
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000022 869da80a 00000002 0000007b 0002 USER STARTED 001 noDelay3_u
.....................
164 Chapter 7. Basic memory management
--> neon allocRgn_u 20000022 869da80a 2 7b 0x66666666 0x6000 0
started aid = 25
--> neon listRegions_u 20000022 869da80a 2 7b
started aid = 25
The actor has 4 regions
start = 0xfffe6000 size = 0x6000 [0x0] (= the new region
start = 0xfffec000 size = 0x6000 [0x2]WR
start = 0xffff3000 size = 0x9000 [0x2]WR
start = 0xffffc000 size = 0x3000 [0x1]EX
-->
The kernel has selected the address and we observe that the region is not
writable.
In the second call, a correct address is transmitted (page aligned and not
overlapping existing regions) without using the K ANYWHERE ag. We use the
K WRITABLE and K FILLZERO attributes (values 1 and 8).

--> neon allocRgn_u 20000022 869da80a 2 7b 0xd0000000 0x5000 1 8


started aid = 25
--> neon listRegions_u 20000022 869da80a 2 7b
started aid = 25
The actor has 5 regions
start = 0xd0000000 size = 0x5000 [0x2]WR (= new region should be FZ too
start = 0xfffe6000 size = 0x6000 [0x0]
start = 0xfffec000 size = 0x6000 [0x2]WR
start = 0xffff3000 size = 0x9000 [0x2]WR
start = 0xffffc000 size = 0x3000 [0x1]EX
-->

The attributes of the new region are incorrect (the FZ ag does not appear). We
encountered various other errors concerning the management of the attributes
during our tests, as we will demonstrate later.
We now request the creation of a region to which access is reserved to threads
executing within the supervisor mode by using the K SUPERVISOR attribute (we
also use the K ANYWHERE ag):
--> neon allocRgn_u 20000022 869da80a 2 7b 0 0x3000 0 3
started aid = 25
--> neon listRegions_u 20000022 869da80a 2 7b
started aid = 25
The actor has 6 regions
start = 0xd0000000 size = 0x5000 [0x2]WR
start = 0xfffe3000 size = 0x3000 [0x4]SU (=the new region
start = 0xfffe6000 size = 0x6000 [0x0]
start = 0xfffec000 size = 0x6000 [0x2]WR
start = 0xffff3000 size = 0x9000 [0x2]WR
start = 0xffffc000 size = 0x3000 [0x1]EX
7.4. Creating a new region 165
Finally, we request the creation of a region overlapping an existing one without
using the K ANYWHERE ag. The call fails:
--> neon allocRgn_u 20000022 869da80a 2 7b 0 0xd0002000 0x1000
started aid = 25
error on rgnAllocate: Region overlapping
-->

We now use the modAttr application (see 7.2.5.2) to change protection attributes
of regions. We will note erroneous results provided by the rgnStat primitive.
We turn o the writable attribute of one page in the ve page region starting
at address 0xd0000000:
--> neon modAttr_u 20000022 869da80a 2 7b 0xd0002000 0x1000 r
started aid = 25
-->

When checking the address space of the actor, we get:


--> neon listRegions_u 20000022 869da80a 2 7b
started aid = 26
The actor has 7 regions
start = 0xd0000000 size = 0x2000 [0x2]WR
start = 0xd0002000 size = 0x3000 [0x2]WR
[
......... 5 other regions] ......
-->

This result is wrong: we should have got:


--> neon listRegions_u 20000022 869da80a 2 7b
started aid = 26
The actor has 8 regions
start = 0xd0000000 size = 0x2000 [0x2]WR (= still writable
start = 0xd0002000 size = 0x1000 [0x0] (= no more writable
start = 0xd0003000 size = 0x2000 [0x2]WR (= still writable
[
......... 5 other regions] ......
-->

We checked on a dedicated example ($CHORUS/memory/bug/bug.c) that in fact


the initial region e ectively split into three regions and that the middle one is
no longer writable: trying to write there provokes an exception.
b) Regions in a supervisor actor
We now use the command to create regions in the supervisor c actor that we
created on the same system and which is identi ed by 23:
--> rsh neon arun cs -la 23
.......................
20000027 869da80a 00000017 00000000 0023 SUP STARTED 001 noDelay3_s.r
.......................
-->
166 Chapter 7. Basic memory management
We rst try to create a region starting at an address which does not belong to
the range of addresses of the supervisor address space:
--> neon allocRgn_u 20000027 869da80a 17 0 0xd0003000 0x5000 1
started aid = 25
error on rgnAllocate: Address space violation
-->

In the next call, performed using the K ANYWHERE ag and the same wrong
address as in the previous one, the kernel selects an address in the supervisor
space:
--> neon allocRgn_u 20000027 869da80a 17 0 0xd0003000 0x5000 0 1
started aid = 25
--> neon listRegions_u 20000027 869da80a 17 0
started aid = 25
The actor has 4 regions
start = 0x7bbce000 size = 0x5000 [0x2]WR (=
the new region
start = 0x7bbe6000 size = 0x2000 [0x6]WR SU
start = 0x7bbe9000 size = 0x9000 [0x6]WR SU
start = 0x7bbf2000 size = 0x3000 [0x5]EX SU
-->

7.5 Freeing regions in an actor


First of all, di erent levels must to be considered:
 the logical level: this concerns the logical address range and therefore the
logical pages;
 the physical level: this concerns the physical pages.
Freeing a logical region does not necessarily mean that the corresponding physi-
cal pages are freed, as regions may be physically shared between di erent actors
(see rgnMapFromActor in 7.6.2.1 and rgnDup in 15.4.1). Pages of physical re-
gions are freed once they are no longer being used by any actors.

7.5.1 Implicit deallocation at actor termination


First of all, when a user actor terminates, all its addresses are freed as they
they are only meaningful in the context of that actor. The physical pages will
be freed when the last actor using the corresponding region(s) terminates (or
frees them explicitly).
An address range of a supervisor actor will be freed at actor termination only if
no other actor is currently sharing it. The logical range and the physical pages
are freed at the same time.
7.5. Freeing regions in an actor 167
A special case occurs for regions which are created with the K SVACTOR ag:
they can only be freed explicitly (by calling the rgnFree primitive) as no actor
owns them. The associated physical pages are freed immediately.

7.5.2 The rgnFree primitive


A region, a subregion or a set of regions/subregions may be explicitly deallocated
by a call to the function
#include <mem/chMem.h>
int rgnFree(
KnCap *actorCap,
KnRgnDesc *rgnDesc
);

This call requests freeing the address space within the range of addresses start-
ing at rgnDesc->startAddr (which must be page-aligned) and whose size is
rgnDesc->size in the actor whose capability is pointed to by actorCap. The
value of size may be rounded up in order to be page-aligned. Contrary to
the memory deallocation by free where the deallocated area must t an area
allocated by a call to malloc exactly, the rgnFree primitive may be used to
deallocate any set of contiguous addresses independently from the way they
were allocated. Furthermore, the corresponding range of addresses may contain
invalid addresses.
As stated earlier, deallocating memory in an actor does not necessarily imply
that the corresponding physical memory is deallocated (that will only be the
case when all the corresponding logical addresses have been deallocated).
The only values of the options eld which are recognized are 0 and the special
K FREEALL value which is a request for deallocating all the regions of the actor.
The actorCap parameter may have the special K SVACTOR value: this value al-
lows regions in the supervisor address space to be deallocated without specifying
a speci c actor capability. This value should only be used for deallocating re-
gions which are not attached to a speci c actor. When actorCap is K SVACTOR,
the K FREEALL value of the options eld cannot be used.
It is important to realize that this type of operation could lead:
 either to the deletion of complete regions and shrinkage of regions as the
following diagram illustrates:
168 Chapter 7. Basic memory management

region 1 startAddr

e ect of rgnFree:
region 2 - region 2 is deallocated
- regions 1 and 3 shrink
startAddr + size
region 3

 or to the partition of a region into two regions (and creation of an unallocated


range of addresses):

startAddr
deallocated address space
startAddr + size inside a region

The function returns K OK when successful, or a negative error code otherwise:


return value error type
K EINVAL inconsistent actor capability or invalid options eld
K EUNKNOWN unknown actor
K EFAULT address out of address space
K EROUND starting address not page-aligned
K ENOMEM system out of resources

7.5.3 Example
We give below the source of a command to externally deallocate regions in a
given actor:
--> cat $CHORUS/sources/memory/rgnFree/rgnFree.c
#include "utilities.h"
KnRgnDesc rgnDesc;
KnCap targetActorCap;
int result;
main(int argc, char *argv[ ]) {
readCap(argv + 1, &targetActorCap);
sscanf(argv[5], "%x", &rgnDesc.startAddr);
sscanf(argv[6], "%x", &rgnDesc.size);
7.5. Freeing regions in an actor 169
result = rgnFree(&targetActorCap, &rgnDesc)
if(result != K_OK)
fprintf(stderr, "error on rgnFree: %s\n", strSysError(result));
else
fprintf(stderr, "rgnFree succeeded\n");
}
-->

We killed the existing actors and restarted a new user c actor for the noDelay3
application and allocated a new region of 6 pages at the address 0xbbbb0000 in
its address space:
--> neon-n noDelay3_u&
[1] 2708
--> started aid = 24

--> rsh neon arun cs | grep noDelay3


started aid = 23
20000029 869da80a 00000018 00000000 0024 USER STARTED 001 noDelay3_u
--> neon allocRgn_u 20000029 869da80a 18 0 0xbbbb0000 0x6000 1
started aid = 23
--> neon listRegions_u 20000029 869da80a 18 0
started aid = 23
The actor has 4 regions
start = 0xbbbb0000 size = 0x6000 [0x2]WR
start = 0xfffec000 size = 0x6000 [0x2]WR
start = 0xffff3000 size = 0x9000 [0x2]WR
start = 0xffffc000 size = 0x3000 [0x1]EX
-->

We deallocate two pages in the middle of the region of that actor:


--> neon rgnFree_u 20000029 869da80a 18 0 0xbbbb2000 0x2000
started aid = 23
rgnFree succeeded
--> neon listRegions_u 20000029 869da80a 18 0
started aid = 23
The actor has 5 regions
start = 0xbbbb0000 size = 0x2000 [0x2]WR
start = 0xbbbb4000 size = 0x2000 [0x2]WR
start = 0xfffec000 size = 0x6000 [0x2]WR
start = 0xffff3000 size = 0x9000 [0x2]WR
start = 0xffffc000 size = 0x3000 [0x1]EX
-->

These results show that the region starting at address 0xbbbb0000 has shrunk
to a rst region of 2 pages, a hole of 2 pages and another region of 2 pages.
170 Chapter 7. Basic memory management
7.6 Advanced operations on regions
7.6.1 Introduction
The service provided by rgnAllocate allocates memory in the address space
of an actor, or possibly in the supervisor address space. At the low level, in
the regions kernel list, this operation may correspond either to the creation of
a new region, to the extension of an existing region or even to the merge of
existing regions into a larger single one.
Another type of service allows the creation of a region in an actor which will be
related to an existing region of another actor. This relation may be:
 strong: a new region is created which will correspond to the same physical
data as an existing one. The two regions allow the two actors to share data. The
two regions may or not correspond to the same virtual address in the address
space of the actors. This is quite similar to the following mechanisms of the
Unix system:
- the sharing of the code segment by two Unix processes after a fork op-
eration. The new process executes the same program, the code region
corresponds to the same virtual address in the new process and is associ-
ated to the same physical area;
- the ability to share data through shared memory segments. In this case,
the two regions will not necessarily correspond to the same virtual address
in the address space of the actors;
 weak: a new region is created which will correspond to a di erent physical
area but will be initialized, when created, with the contents of the existing re-
gion. This mechanism is similar to the duplication of the data region of a Unix
process when it forks.
Depending on the active memory management model, the following services are
provided:
 within the three memory management models, sharing memory: the corre-
sponding service is accessed through the rgnMapFromActor primitive;
 within the MEM PROTECTED and MEM VIRTUAL memory models, allo-
cation of memory with initialization of the allocated space from existing regions
through the rgnInitFromActor primitive;
 with the MEM VIRTUAL memory management model, duplicating a com-
plete address space without a per-region option for copying or sharing through
the rgnDup primitive.
These di erent services are presented in chapter 15.
7.6. Advanced operations on regions 171
7.6.2 Sharing memory
7.6.2.1 The rgnMapFromActor primitive
The basic service allowing actors to share memory through regions of their
address space is accessed by calling the primitive
#include <mem/chMem.h>
int rgnMapFromActor(
KnCap *targetActorCap,
KnRgnDesc *targetRgnDesc,
KnCap *sourceActorCap,
KnRgnDesc *sourceRgnDesc
);

which is provided for the three memory management models.


In this type of call:
 sourceActorCap points to the capability of a source actor whose address space
is used for retrieving existing regions. As usual the K MYACTOR value designates
the current actor;
 targetActorCap points to the capability of the target actor where memory
will be allocated. The K MYACTOR value may be used for naming the current
actor. The K SVACTOR value corresponds to a request for allocating memory in
the supervisor address space independently from any actor, remember that the
corresponding region(s) will have to be freed explicitly by calling rgnFree;
 targetRgnDesc points to a region descriptor which de nes the address range
in the actor identi ed by targetActorCap that is concerned by the operation. In
the corresponding descriptor:
- startAddr de nes the starting address. It is important if the K ANYWHERE
ag is not set in the options eld and must then be page-aligned;
- size de nes the size of the range of addresses. If needed, it will be
rounded-up by the system to be page-aligned;
- in the options eld K FILLZERO will be ignored;
 the only signi cant eld in the region's descriptor pointed to by sourceRgn-
Desc is sourceAddr. This eld speci es the rst address of the address range
of the actor pointed to by sourceActorCap which are mapped in the target ac-
tor. The size of this address range is de ned by the size eld of the target
region's descriptor. It is important to note that the corresponding range of
addresses in the source actor may not contain holes (all the addresses
must be in the scope of an allocated region of this actor).
172 Chapter 7. Basic memory management
If the call succeeds, all regions in the address range of the source actor which
starts at sourceRgnDesc->startAddr (whose size is targetRgnDesc->size) are
mapped into the address range of the target actor which starts at the address
targetRgnDesc->startAddr.
The e ect of this call is shown in the following diagram, where the mapped
(sub)regions of the source actor and the new regions of the target actors corre-
spond to the same physical memory areas:
targetRgnDesc- >startAddr
READABLE REGION
READABLE REGION
>startAddr
sourceRgnDesc- WRITABLE REGION >size
targetRgnDesc-

WRITABLE REGION READABLE REGION

READABLE REGION

source actor target actor


The function returns K OK if the call succeeds, otherwise a negative error code
is returned:
return value error type
K EINVAL inconsistent actor capability
K EUNKNOWN unknown actor
K EROUND a startAddr eld is not page-aligned
K EFAULT address out of address space
K EROUND starting address not page-aligned
K ENOMEM system out of resources
K ESPACE region outside legal address space or size is 0
K EOVERLAP target address range overlapping an existing region
K ESIZE one of the K WRITABLE, K READABLE or K EXECUTABLE
ags is incompatible with the attributes of the source region
K EADDR the source address range contains holes
7.6.2.2 Example
First we give the code of an application which:
 creates a new region of two pages which is mapped at the 0xb0000000 in its
virtual address space if the actor is a user actor, or at a non-speci ed address
(K ANYWHERE) if it is a supervisor actor. The physical address 1 corresponding
to the rst logical page is printed;
 writes the character X in every byte of this region;
 waits for one minute and then, displays the contents of the rst 16 bytes of
the two pages of this region.
7.6. Advanced operations on regions 173
-1-> cat $CHORUS/sources/memory/map/source/sourceMap.c
#include "utilities.h"
#include <mem/chMem.h>
KnRgnDesc rgnDesc;
KnCap actorCap;
int ind, result;
char *charPointer;
KnTimeVal delay;
KnActorPrivilege privilege;
PhAddr physAddr;

main(int argc, char *argv[ ]) {


rgnDesc.size = 0x2000; /* two pages will be allocated */
rgnDesc.options = K_WRITABLE;
rgnDesc.startAddr = 0xb0000000; /* starting address imposed if user actors */
result = actorPrivilege(K_MYACTOR, &privilege, NULL);
if(result < 0) {
fprintf(stderr, "error on actorPrivilege: %s\n", strSysError(result));
exit(1); }
if (privilege == K_SUPACTOR)
rgnDesc.options |= K_ANYWHERE;
result = rgnAllocate(K_MYACTOR, &rgnDesc);
if(result < 0) {
fprintf(stderr, "error on rgnAllocate: %s\n", strSysError(result));
exit(1); }
vmPhysAddr(K_MYACTOR, rgnDesc.startAddr, &physAddr);
actorSelf(&actorCap);
fprintCap(stderr, "A region has been allocated in actor", &actorCap);
fprintf(stderr, " logical address: %p\n", rgnDesc.startAddr);
fprintf(stderr, " physical address: %p\n", physAddr);
/* the region is filled with the character X */
charPointer = (char *) rgnDesc.startAddr;
for(ind = 0; ind < 0x2000; ind ++) {
*charPointer = 'X';
charPointer ++; }
K_MILLI_TO_TIMEVAL(&delay, 60000); threadDelay(&delay);
/* when awaking, after 1 minute, display the contents of the region */
charPointer = (char *) rgnDesc.startAddr;
printf("First page: ");
for(ind = 0; ind < 0x10; ind ++)
putchar(charPointer[ind]);
putchar('\n');
charPointer = (char *) rgnDesc.startAddr + 0x1000;
printf("Second page: ");
for(ind = 0; ind < 0x10; ind ++)
putchar(charPointer[ind]);
putchar('\n');
}
174 Chapter 7. Basic memory management
-1-> neon sourceMap_u
started aid = 23
A region has been allocated in actor: 20000042 869da80a 17 1df
logical address: 0xb0000000
physical address: 0xebd000
(= Now, sleep for one minute
The next source corresponds to a command which:
 maps one page of the region of a given actor at the 0xc0000000 address in
its address space if it is a user actor, or at a non-speci ed address if it is a
supervisor actor. The source actor's capability and the starting address of the
region in that actor are passed as arguments to the command. It prints the
physical address 2 corresponding to that logical page;
 creates a one page region that is contiguous to the previous one;
 displays the attributes of all its regions;
 writes the character Y in the two pages starting at 0xc0000000.
-2-> cat $CHORUS/sources/memory/map/target/targetMap.c
#include "utilities.h"
#include <mem/chMem.h>
KnRgnStat statArray[10];
KnRgnDesc targetRgnDesc, sourceRgnDesc;
KnCap sourceActorCap;
int ind, result;
char *charPointer;
KnActorPrivilege privilege;
PhAddr physAddr;

main(int argc, char *argv[ ]) {


readCap(argv + 1, &sourceActorCap); /* actor owning the mapped region */
/* starting address of the source address range */
sscanf(argv[5], "%x", &sourceRgnDesc.startAddr);
result = actorPrivilege(K_MYACTOR, &privilege, NULL);
if(result < 0) {
fprintf(stderr, "error on actorPrivilege: %s\n", strSysError(result));
exit(1);
}
/* description of the target address range */
targetRgnDesc.startAddr = 0xc0000000; /* starting address is imposed */
targetRgnDesc.size = 0x1000; /* one page of source region is mapped */
targetRgnDesc.options = K_WRITABLE;
if (privilege == K_SUPACTOR)
targetRgnDesc.options |= K_ANYWHERE;
/* Requesting for mapping */
result = rgnMapFromActor(K_MYACTOR, &targetRgnDesc,
&sourceActorCap, &sourceRgnDesc);
7.6. Advanced operations on regions 175
if(result < 0) {
fprintf(stderr, "error on rgnMapFromActor: %s\n", strSysError(result));
exit(1);
}
/* get and print physical address of mapped page */
printf("the region has been mapped in the actor's address space\n");
vmPhysAddr(K_MYACTOR, targetRgnDesc.startAddr, &physAddr);
printf(" logical address: %p\n",
charPointer = (char *) targetRgnDesc.startAddr);
printf(" physical address: %p\n", physAddr);
/* read and print the contents of the region */
printf("contents of mapped region: ");
for(ind = 0; ind < 0x10; ind ++)
putchar(charPointer[ind]);
putchar('\n');
/* fill the region with the character Y */
charPointer = (char *) targetRgnDesc.startAddr;
for(ind = 0; ind < 0x1000; ind ++)
charPointer[ind] = 'Y';
/* create a new region contiguous to the mapped one */
/* with the same attributes */
targetRgnDesc.startAddr += targetRgnDesc.size;
targetRgnDesc.options = K_WRITABLE;
targetRgnDesc.size = 0x1000; /* one page of source region is mapped */
result = rgnAllocate(K_MYACTOR, &targetRgnDesc);
if(result < 0)
fprintf(stderr, "error on rgnAllocate: %s\n", strSysError(result));
else {
/* fill the region with the character Y */
charPointer = (char *) targetRgnDesc.startAddr;
for(ind = 0; ind < 0x1000; ind ++)
charPointer[ind] = 'Y';
}
/* get and print attributes of all regions of the actor */
result = doRegionStat(K_MYACTOR, statArray, sizeof(statArray));
printf("The actor has %d regions\n", result);
for(ind = 0; ind < result; ind ++)
printRegion(statArray + ind);
}

While the main thread of the actor executing the sourceMap application is
sleeping, we create a new actor whose main thread executes the targetMap
application. The capability of the source actor and the 0xb0000000 address
corresponding to the page to be mapped in the new actor are passed as argu-
ments.
176 Chapter 7. Basic memory management
-2-> neon targetMap_u 20000042 869da80a 17 1df 0xb0000000
started aid = 22
the region has been mapped in the actor's address space
logical address: 0xc0000000
physical address: 0xebd000
contents of mapped region: XXXXXXXXXXXXXXXX
The actor has 4 regions
start = 0xc0000000 size = 0x2000 [0x2]WR
start = 0xfffea000 size = 0x6000 [0x2]WR
start = 0xffff1000 size = 0x2000 [0x2]WR
start = 0xffff3000 size = 0xc000 [0x1]EX
-2->

When the main thread of the actor corresponding to the sourceMap application
is awoken, it displays the contents of the two pages of the region which has been
partially mapped in the target actor and terminates:
First page: YYYYYYYYYYYYYYYY (= page has been modi ed
Second page: XXXXXXXXXXXXXXXX
-1->

These results show that the rst page of the region of the rst actor has been
mapped to the second one: the corresponding physical addresses 1 and 2
are equal (0xebd000). Therefore, when the main thread of the source actor is
awoken, it can see the modi cation to the contents of this page. On the other
hand, it does not see the modi cations to the second page: this page has not
been mapped in the target actor. We can also note that the two contiguous
regions which were created in the target actor's address space have been merged
though they do not have the same attributes, this is due to the fact that one is
mapped and the other one is not. The following diagram illustrates this memory
sharing through the regions in the actors:
physical address: 0xebd000

0xb0000000
0xb0001000
0xb0002000
source actor

0xc0000000
0xc0001000
0xc0002000

target actor physical memory


7.6. Advanced operations on regions 177
The same operations have also been performed using various combinations of
actors with di erent privileges:
 a supervisor source actor and a user target actor:
-1-> neon sourceMap_s.r
started aid = 23
A region has been allocated in actor: 20000046 869da80a 17 1
logical address: 0x7bbe1000
physical address: 0xead000
-2-> neon targetMap_u 20000046 869da80a 17 1 0x7bbe1000
started aid = 22
the region has been mapped in the actor's address space
logical address: 0xc0000000
physical address: 0xead000
contents of mapped region: XXXXXXXXXXXXXXXX
The actor has 4 regions
start = 0xc0000000 size = 0x2000 [0x2]WR
start = 0xfffea000 size = 0x6000 [0x2]WR
start = 0xffff1000 size = 0x2000 [0x2]WR
start = 0xffff3000 size = 0xc000 [0x1]EX
-2->
First page: YYYYYYYYYYYYYYYY (= page has been modi ed
Second page: XXXXXXXXXXXXXXXX
-1->
 a user source actor and a supervisor target actor:
-1-> neon sourceMap_u
started aid = 23
A region has been allocated in actor: 20000048 869da80a 17 20f
logical address: 0xb0000000
physical address: 0xebf000
-2-> neon targetMap_s.r 20000048 869da80a 17 20f 0xb0000000
started aid = 22
the region has been mapped in the actor's address space
error on rgnAllocate: Region overlapping
logical address: 0x7bbe0000
physical address: 0xebf000
contents of mapped region: XXXXXXXXXXXXXXXX
The actor has 4 regions
start = 0x7bbde000 size = 0x2000 [0x6]WR SU
start = 0x7bbe0000 size = 0x1000 [0x2]WR
start = 0x7bbe5000 size = 0x2000 [0x6]WR SU
start = 0x7bbe7000 size = 0xc000 [0x5]EX SU
-2->
First page: YYYYYYYYYYYYYYYY (= page has been modi ed
Second page: XXXXXXXXXXXXXXXX
-1->
178 Chapter 7. Basic memory management
 a supervisor source actor and a supervisor target actor:
-1-> neon sourceMap_s.r
started aid = 23
A region has been allocated in actor: 2000004a 869da80a 17 21c
logical address: 0x7bbe1000
physical address: 0xf47000
-2-> neon targetMap_s.r 2000004a 869da80a 17 21c 0x7bbe1000
started aid = 22
the region has been mapped in the actor's address space
error on rgnAllocate: Region overlapping
logical address: 0x7bbe5000
physical address: 0xf47000
contents of mapped region: XXXXXXXXXXXXXXXX
The actor has 4 regions
start = 0x7bbcf000 size = 0x2000 [0x6]WR SU
start = 0x7bbd5000 size = 0xc000 [0x5]EX SU
start = 0x7bbe5000 size = 0x1000 [0x2]WR
start = 0x7bbe6000 size = 0x2000 [0x6]WR SU
-2->
First page: YYYYYYYYYYYYYYYY (=page has been modi ed
Second page: XXXXXXXXXXXXXXXX
-1->

These results show that the same region may be simultaneously associated to
regions in the supervisor address space and in the virtual address space of
user actors (that is in the user address space). Finally, we observe failures of
rgnAllocate calls due K EOVERLAP errors.

7.6.3 Inheriting a copy of existing regions


7.6.3.1 The rgnInitFromActor primitive
Here the purpose is to create regions in an actor and to initialize those regions
from the contents of existing regions in another actor, thus the target actor
inherits copies of the source regions.
This service is provided by the VIRTUAL ADDRESS SPACE feature when calling
the primitive
#include <mem/chMem.h>
int rgnInitFromActor(
KnCap *targetActorCap,
KnRgnDesc *targetRgnDesc,
KnCap *sourceActorCap,
KnRgnDesc *sourceRgnDesc
);
7.6. Advanced operations on regions 179
The arguments, their constraints and their interpretation are the same as for the
rgnMapFromActor primitive. In particular, the source region may not contain
holes. An additional requirement is that the size eld of sourceRgnDesc
must be initialized and must be less or equal to the size eld of
targetRgnDesc.
When the call succeeds, K OK is returned, otherwise one of the following error
value is returned:
return value error type
K EINVAL inconsistent actor capability
K EUNKNOWN unknown actor
K EROUND a eld startAddr is not page-aligned
K EFAULT address out of address space
K EROUND one of the start addresses is not page-aligned
K ENOMEM system out of resources
K ESPACE region outside legal address space or size is 0
K EOVERLAP target address range overlapping an existing region
or K ANYWHERE or K RESTRICTIVE and not enough room
K ESIZE sourceRgnDesc -> size > targetRgnDesc -> size
K EADDR the source address range contains holes
7.6.3.2 Example
First, we change the sequence of the targetMap.c source
/* Request for mapping */
result = rgnMapFromActor(K_MYACTOR, &targetRgnDesc,
&sourceActorCap, &sourceRgnDesc);
if(result < 0) {
fprintf(stderr, "error on rgnMapFromActor: %s\n", strSysError(result));
exit(1);
}
printf("the region has been mapped in the actor's address space\n");

into its equivalent that calls rgnInitFromActor and we change the message
displayed when the new region has been created. Using the corresponding
targetInit u command:
/* Request for initializing a new region */
result = rgnInitFromActor(K_MYACTOR, &targetRgnDesc,
&sourceActorCap, &sourceRgnDesc);
if(result < 0) {
fprintf(stderr, "error on rgnInitFromActor: %s\n", strSysError(result));
exit(1);
}
printf("initialized region created in the actor's address space\n");

Then, we create the source and the target actors:


180 Chapter 7. Basic memory management
-1-> neon sourceMap_u
started aid = 23
A region has been allocated in actor: 2000004c 869da80a 17 220
logical address: 0xb0000000
physical address: 0xeaf000
-2-> neon targetInit_u 2000004c 869da80a 17 220 0xb0000000
started aid = 22
error on rgnInitFromActor: Inconsistent size arguments
-2->
First page: XXXXXXXXXXXXXXXX
Second page: XXXXXXXXXXXXXXXX
-1->

We get a K ESIZE error on the rgnInitFromActor call because the size eld
of the source region's descriptor is zero. After having initialized its value to
0x1000 (which corresponds to the page size) before calling rgnInitFromActor

sourceRgnDesc.size = 0x1000;

we create the two actors:


-1-> neon sourceMap_u
neon sourceMap_u
started aid = 23
A region has been allocated in actor: 20000050 869da80a 17 258
logical address: 0xb0000000
physical address: 0xedb000
-2-> neon targetInit_u 20000050 869da80a 17 258 0xb0000000
started aid = 22
initialized region created in the actor's address space
logical address: 0xc0000000
physical address: 0xeae000
contents of mapped region: XXXXXXXXXXXXXXXX
The actor has 4 regions
start = 0xc0000000 size = 0x2000 [0x2]WR
start = 0xfffea000 size = 0x6000 [0x2]WR
start = 0xffff1000 size = 0x2000 [0x2]WR
-2->
First page: XXXXXXXXXXXXXXXX (=page is not modi ed
Second page: XXXXXXXXXXXXXXXX
-1->

We can observe from these results that:


 the region created when rgnInitFromActor is called has the same initial con-
tents as the current contents of the source region, except physical addresses are
di erent. Therefore, the new region is a physical copy of the source region and
the modi cations apply to the physical memory area of the new region;
7.7. Manipulating supervisor address space 181
 this region is merged with the contiguous region that is then created by
rgnAllocate ; the regions are contiguous and have the same attributes.
The following gure illustrates the association between logical pages and phys-
ical pages:

0xb0000000
0xb0001000
0xb0002000
source actor

0xc0000000
0xc0001000
0xc0002000

target actor physical memory

7.7 Manipulating supervisor address space


7.7.1 Introduction
In this section we describe speci c services provided for managing the supervi-
sor address space.
We rst recall that the supervisor address space is shared by all supervisor ac-
tors. One address in the supervisor address space designates the same physical
object in all supervisor actors, all threads executing in a supervisor actor may
access all allocated addresses in this space freely. Allocation of space in the
supervisor address space can be performed as follows:
 by allocating a new region when calling rgnAllocate with the capability of
a supervisor actor passed as rst argument. The region will be automatically
freed when this actor is deleted (if the region has not been duplicated);
 by allocating a new region by calling rgnAllocate with the K SVACTOR ag.
In this case, the region will not be attached to any (supervisor) actor and will
have to be explicitly deleted by calling rgnFree;
 by independently allocating pages using the speci c svPagesAllocate prim-
itive. In this case, the pages will have to be explicitly freed by calling the
svPagesFree primitive. If an actor is deleted without calling that primitive for
all the pages it allocated by calling svPagesAllocate, the pages will remain
allocated and will still be part of the allocated supervisor address space.
182 Chapter 7. Basic memory management
We will also present here the speci c service provided for copying the contents
of an area of the supervisor address space into another one (svMemWrite and
svMemRead) without providing exceptions.
The presentation of speci c functions allowing the copy of an area of the su-
pervisor address space to an area of the address space of a user actor or the
converse will be covered in the section on the local access points in the context
of which these operations are commonly used.
7.7.2 Displaying all regions in supervisor address space
We rst produce a utility which displays all the ranges of addresses which are
currently allocated in the supervisor address space. It displays not only the
regions which are part of the address space of a supervisor actor, but also those
which have been created independently of all actors. In fact the information
which is displayed is exactly the same as the memory information displayed by
the cs utility for the kern actor (local identi er 1).
The command is derived from the listRegions command we used in 7.2.7.2,
using K SVACTOR as rst argument of the rgnStat primitive:
--> cat $CHORUS/sources/memory/supRegions/supRegionsStat.c
#include <stdio.h>
#include <chorus.h>
#define REGIONS_MAX_NUMBER 1000
KnRgnStat statArray[REGIONS_MAX_NUMBER];
main( ) {
int ind, result;
result = doRegionStat(K_SVACTOR, statArray, REGIONS_MAX_NUMBER);
printf("Number of regions in supervisor address space: %d\n", result);
if(result > REGIONS_MAX_NUMBER)
result = REGIONS_MAX_NUMBER;
for(ind = 0; ind < result; ind ++)
printRegion(statArray+ind);
}

When this is executed by a supervisor c actor, we get the following:


--> neon supRegionsStat_s.r
started aid = 23
Number of regions in supervisor address space: 58
start = 0x0 size = 0xa0000 [0x6]WR SU
start = 0x100000 size = 0xf00000 [0x6]WR SU
start = 0x40000000 size = 0x3000 [0x5]EX SU
[ ]
.......... more lines ...............
start = 0x7bcd9000 size = 0xb5000 [0x0]
start = 0x7bd8e000 size = 0x4271000 [0x6]WR SU
-->
7.7. Manipulating supervisor address space 183
7.7.3 The supervisor address space memory allocator
The three memory management models provide an interface for allocating and
freeing memory in the supervisor address space. This allocator is similar to the
malloc/free services of the standard C library. The last region in the previous
results corresponds to a heap where pages are allocated by the supervisor address
space allocator.
Memory allocation is done by calling the primitive
#include <mem/chMem.h>
int svPagesAllocate(
VmAddr *addr,
VmSize size,
VmFlags ags,
KnCap *actorCap
);

In this type of call:


 size speci es the number of bytes to be allocated. This number will be rounded
to get a page-aligned number (the number passed as argument need not to be
page-aligned);
 the ags parameter is a combination of the following values:
- K NOWAITFORMEMORY: the call will fail if there is not enough physical mem-
ory available for allocating the pages. If this ag is not set, the call will
block until there is enough memory;
- K REDALLOC: this ag is only used by the operating system;
 actorCap speci es the actor to which the allocated memory is associated.
This association does not mean that this actor will own the allocated space,
this space will not be freed when the actor is deleted. This association is used
by functions such as actorRestart, for example. The K SVACTOR value may be
used if the association is not needed.
When returning from the call, the variable pointed to by addr will contain the
virtual address of the allocated memory.
The primitive may be called from an interrupt handler, in that case the K NO-
WAITFORMEMORY must be set.
A call to the function returns K OK if the allocation was successful, otherwise a
negative error code is returned:
184 Chapter 7. Basic memory management
return value error type
K EINVAL primitive called from an interrupt handler and the
K NOWAITFORMEMORY was not set
K EUNKNOWN unknown actor
K EFAULT address out of address space
K ENOMEM system out of memory
K ESIZE the size argument is 0
Deallocation of a virtual address range in the supervisor address space may be
performed by calling the primitive
#include <mem/chMem.h>
int svPagesFree(
VmAddr addr,
VmSize size,
KnCap *actorCap
);
This call deallocates the address range de ned by addr and size. This ad-
dress range must be a sub-range of a range previously allocated by calling
svPagesAllocate. The value of addr must be paged-aligned and the value of
size will be automatically page-aligned by the kernel. The value of actorCap
allows the association of the allocated region with the actor to be cleared: this
value must match the value which was de ned when the space was allocated.
The function returns K OK if the call was successful and a negative value other-
wise (the K EROUND value is returned if addr in not page-aligned).
--> cat $CHORUS/sources/memory/superAlloc/superAlloc.c
#include "utilities.h"
KnRgnStat statArray[20];
int result, ind;
VmAddr addr;
KnCap actorCap;

main(int argc, char *argv[ ]) {


actorSelf(&actorCap);
fprintCap(stdout, "actor's capability", &actorCap);
result = doRegionStat(K_MYACTOR, statArray, 20);
printf("actor has %d regions\n", result);
for(ind = 0; ind < result; ind ++)
printRegion(statArray + ind);
/* allocate 4 pages in the current actor */
result = svPagesAllocate(&addr, 0x4000, 0, K_MYACTOR);
if(result < 0) {
printf("error on svPagesAllocate: %s\n", strSysError(result));
exit(2);
}
7.7. Manipulating supervisor address space 185
printf("corresponding address : %p\n", addr);
/* write a character into the first byte of each page */
/* 'a' in 1st page, 'b' in 2nd, and so on */
for (ind = 0; ind < 4; ind ++)
*((char *) addr + ind * 0x1000) = 'a' + ind;
result = doRegionStat(K_MYACTOR, statArray, 20);
printf("actor has %d regions\n", result);
for(ind = 0; ind < result; ind ++)
printRegion(statArray + ind);
}
--> neon superAlloc_s.r
started aid = 23
actor's capability: 20000056 869da80a 17 0
actor has 3 regions
start = 0x7bbe3000 size = 0x2000 [0x6]WR SU
start = 0x7bbe8000 size = 0x1000 [0x6]WR SU
start = 0x7bbe9000 size = 0xa000 [0x5]EX SU
corresponding address : 0x7fdd6000
actor has 3 regions
start = 0x7bbe3000 size = 0x2000 [0x6]WR SU
start = 0x7bbe8000 size = 0x1000 [0x6]WR SU
start = 0x7bbe9000 size = 0xa000 [0x5]EX SU
--> neon superAlloc_s.r
started aid = 23
actor's capability: 20000057 869da80a 17 1
actor has 3 regions
start = 0x7bbe3000 size = 0x2000 [0x6]WR SU
start = 0x7bbe8000 size = 0x1000 [0x6]WR SU
start = 0x7bbe9000 size = 0xa000 [0x5]EX SU
corresponding address : 0x7fdd2000
actor has 3 regions
start = 0x7bbe3000 size = 0x2000 [0x6]WR SU
start = 0x7bbe8000 size = 0x1000 [0x6]WR SU
start = 0x7bbe9000 size = 0xa000 [0x5]EX SU
-->

These results show that the address associated to the four allocated pages in
the supervisor address space does not belong to the address range of any re-
gion of the actor. Corresponding addresses are part of the supervisor address
space and will not be reallocated until they have been explicitly freed by calling
svPagesFree.
The next application displays the contents of the pages previously allocated in
the supervisor address space (an address is passed as fth argument) and frees
the allocated pages (the capability of the creating actor is passed as four rst
arguments):
186 Chapter 7. Basic memory management
--> cat $CHORUS/sources/memory/superFree/superFree.c
#include "utilities.h"
char *addr; int result, ind;
KnCap actorCap;
main(int argc, char *argv[ ]) {
readCap(argv + 1, &actorCap);
sscanf(argv[5], "%p", &addr);
for(ind = 0; ind < 4; ind ++)
printf("%p ==> %c\n", addr + ind * 0x1000, *(addr +ind * 0x1000));
result = svPagesFree((VmAddr)addr, 0x4000, &actorCap);
if(result < 0)
fprintf(stderr, "error on svPagesFree: %s\n", strSysError(result));
}
-->

We rst use the command for the rst actor we executed, we can check that the
space is still allocated although the actor is terminated:
--> neon superFree_s.r 20000056 869da80a 17 0 0x7fdd6000
started aid = 23
0x7fdd6000 ==> a
0x7fdd7000 ==> b
0x7fdd8000 ==> c
0x7fdd9000 ==> d
-->

A second call to free the same space in the supervisor address space provokes
an exception:
--> neon superFree_s.r 20000056 869da80a 17 0 0x7fdd6000
started aid = 23
Segmentation fault thread 7 PC 7bbe8067 faultAddr 7fdd6000

We now free the space allocated by the second actor, but using the capability
of the rst one: the pages are freed (the capability is not needed to free the
space):
--> neon superFree_s.r 20000056 869da80a 17 0 0x7fdd2000
started aid = 23
0x7fdd2000 ==> a
0x7fdd3000 ==> b
0x7fdd4000 ==> c
0x7fdd5000 ==> d
-->

We now request allocation of space in the supervisor address space. We can


check that the supervisor allocator reuses previously freed addresses:
--> neon superAlloc_s.r
started aid = 23
actor's capability: 2000005b 869da80a 17 1
7.7. Manipulating supervisor address space 187
actor has 3 regions
start = 0x7bbe3000 size = 0x2000 [0x6]WR SU
start = 0x7bbe8000 size = 0x1000 [0x6]WR SU
start = 0x7bbe9000 size = 0xa000 [0x5]EX SU
corresponding address : 0x7fdd6000 (= same as in rst call
actor has 3 regions
start = 0x7bbe3000 size = 0x2000 [0x6]WR SU
start = 0x7bbe8000 size = 0x1000 [0x6]WR SU
start = 0x7bbe9000 size = 0xa000 [0x5]EX SU
-->

7.7.4 Safely accessing supervisor address space


The svMemRead and svMemWrite functions are available within the di erent
memory management modules and may be used by threads executing with the
supervisor privilege to access the supervisor address space safely.
Safely means that if an invalid access to a protected address or to an invalid
address takes place during the copy, the corresponding exception will be caught
and the call will return an error. Performing the same operation using a function
like bcopy (or memcpy) would lead to a termination of the calling actor due to
the exception.
7.7.4.1 The svMemRead primitive
A call to the function
#include <mem/chMem.h>
int svMemRead(
void *srcAddr,
void *dstAddr,
int count
);

copies count bytes from the address srcAddr in the supervisor address space to
the address dstAddr in the supervisor address space. The function returns K OK
if the call succeeds.
It returns the K EFAULT error value if a memory fault occurs during access to
one of the two areas which have srcAddr and dstAddr as the rst address and
count as size.
7.7.4.2 The svMemWrite primitive
A call to the function
188 Chapter 7. Basic memory management
#include <mem/chMem.h>
int svMemWrite(
void *srcAddr,
void *dstAddr,
int count
);

copies count bytes from the address srcAddr in the supervisor address space to
the address dstAddr in the supervisor address space. The function returns K OK
if the call succeeds.
If a memory fault occurs during the operation, the function returns:
 K EFAULT if a memory fault occurred when accessing the destination memory
area;
 K EFAIL if a memory fault occurred when accessing the source memory area.
7.7.4.3 Example
In the next example, which will be executed by supervisor c actors:
 we rst create, two "one page" regions in the supervisor address space. These
are followed in that address space by a non-allocated page: this is ensured by
rst allocating of a 4 page region using a rgnAllocate call and the creation of
holes by two successive calls to rgnFree;
 the source memory area is located at the end of the rst allocated page and
contains sourceLength bytes, this value is retrieved using the atoi(argv[1])
call;
 the destination area is located at the end of the second allocated page and
contains 16 bytes;
 the second argument of the command de nes the count number of bytes that
will be copied from the source area into the target area.
This is illustrated in the following gure:
count
startAddr sourceAddr (argv[2])

sourceLength 16
(argv[1])
startAddr+0x2000 targetAddr

The source code of the command is the following:


7.7. Manipulating supervisor address space 189
--> cat $CHORUS/sources/memory/supSpaceCopy/supSpaceCopy.c
#include <chorus.h>
#include <stdio.h>
KnRgnDesc rgnDesc;

main(int argc, char *argv[ ]) {


int ind, result, count, sourceLength;
char *sourceAddr, *targetAddr;

sourceLength = atoi(argv[1]);
count = atoi(argv[2]);
rgnDesc.options = K_ANYWHERE | K_WRITABLE | K_FILLZERO;
rgnDesc.startAddr = 0x0;
rgnDesc.size = 0x4000;
result = rgnAllocate(K_MYACTOR, &rgnDesc);
fprintf(stderr, "startAddr: %p\n", rgnDesc.startAddr);
sourceAddr = (char *) rgnDesc.startAddr + 0x1000 - sourceLength;
targetAddr = (char *) rgnDesc.startAddr + 0x3000 - 16;
fprintf(stderr, "sourceAddr: %p\n", sourceAddr);
fprintf(stderr, "targetAddr: %p\n", targetAddr);
/* creating two holes after the pages */
rgnDesc.startAddr += 0x1000;
rgnDesc.size = 0x1000;
rgnDesc.options = 0;
result = rgnFree(K_MYACTOR, &rgnDesc);
rgnDesc.startAddr += 0x2000;
result = rgnFree(K_MYACTOR, &rgnDesc);
/* initializing source area */
for(ind = 0; ind < sourceLength; ind ++)
sourceAddr[ind] = 'a' + ind;
/* copying */
result = svMemRead(sourceAddr, targetAddr, count);
/* result = svMemWrite(sourceAddr, targetAddr, count); */
/* bcopy(sourceAddr, targetAddr, count); */
if(result < 0)
fprintf(stderr, "error on svMem: %s\n", strSysError(result));
fprintf(stderr, "after svMemRead addr's contents: %s\n",
targetAddr);
}

We have produced three di erent binaries which are executable in supervisor


mode:
 the rst one (supSpaceCopy1 s.r) uses svMemRead, we give three examples
of its execution:
190 Chapter 7. Basic memory management
--> neon supSpaceCopy1_s.r 20 10
started aid = 23
startAddr: 0x7bbe0000
sourceAddr: 0x7bbe0fec
targetAddr: 0x7bbe2ff0
after svMemRead addr's contents: abcdefghij
--> neon supSpaceCopy1_s.r 12 13
started aid = 23
startAddr: 0x7bbe0000
sourceAddr: 0x7bbe0ff4
targetAddr: 0x7bbe2ff0
error on svMem: Bad address
after svMemRead addr's contents: abcdefghijkl
--> neon supSpaceCopy1_s.r 18 17
started aid = 23
startAddr: 0x7bbe0000
sourceAddr: 0x7bbe0fee
targetAddr: 0x7bbe2ff0
error on svMem: Bad address
Segmentation fault thread 7 PC 7bbf18e1 faultAddr 7bbe3000

 the second one (supSpaceCopy2 s.r) uses svMemWrite, and we give the results
of two erroneous executions:
--> neon supSpaceCopy2_s.r 12 13
started aid = 23
startAddr: 0x7bbe0000
sourceAddr: 0x7bbe0ff4
targetAddr: 0x7bbe2ff0
error on svMem: Bad address
after svMemRead addr's contents: abcdefghijkl
--> neon supSpaceCopy2_s.r 18 17
started aid = 14
startAddr: 0xfdcd3000
sourceAddr: 0xfdcd3fee
targetAddr: 0xfdcd5ff0
error on svMem: Transaction failed
Segmentation fault thread 20 PC fdcddd61 faultAddr fdcd6000

 nally, the third one (supSpaceCopy3 s.r) uses bcopy and we give the results
of two erroneous executions:
--> neon supSpaceCopy3_s.r 12 13
started aid = 23
startAddr: 0x7bbe0000
sourceAddr: 0x7bbe0ff4
targetAddr: 0x7bbe2ff0
Segmentation fault thread 7 PC 7bbf19a3 faultAddr 7bbe1000
7.8. The POSIX shared memory 191
--> neon supSpaceCopy3_s.r 18 17
started aid = 23
startAddr: 0x7bbe0000
sourceAddr: 0x7bbe0fee
targetAddr: 0x7bbe2ff0
Segmentation fault thread 7 PC 7bbf19a3 faultAddr 7bbe3000
-->

7.8 The POSIX shared memory


7.8.1 The POSIX SHM feature and the SHM REP actor
This provides a compatible implementation of the POSIX 1003.1b programming
interface to access shared memory objects.
When this feature is active, a SHM REP user actor is created which is initially
empty. It is used as a repository for shared memory objects. Its address space is
used for creating the objects by calls to rgnAllocate, and actors using shared
memory objects map the corresponding regions to their own address space by
calling rgnMapFromActor. Thus, using Posix shared memory objects does not
use the supervisor address space.
--> rsh neon arun cs
started aid = 22
20000016 869da80a 00000002 00000001 0002 USER STOPPED 000 SHM_REP
--> rsh neon arun cs -la 2
started aid = 22
ChorusOS r4.0.0 Site 0 Time 5m 23
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000016 869da80a 00000002 00000001 0002 USER STOPPED 000 SHM_REP
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
20000016 869da80a def no fac0ac 0 20000016 869da80a
-->

Let us review the main characteristics of the interface:


 a shared memory object has an external name that is a string which does not
appear in the le system. Before using a shared memory object a le descriptor
has to be acquired, the shm open primitive is similar to the open primitive for
opening les. It also allows the creation of a new shared memory object;
 the size of the memory object must be set by calling ftruncate;
 the mapping between a memory object and a c actor's address space is per-
formed by calling the mmap primitive which returns an address for the shared
memory object into the calling actor's address space.
192 Chapter 7. Basic memory management
7.8.2 Creating/opening a shared memory object
The acquisition of a le descriptor for a shared memory object and its creation
are performed by calling the primitive
#include <sys/mman.h>
int shm open(
const char *name,
int o ag,
...
);
When it is successful, the primitive returns a le descriptor for the shared mem-
ory object whose name is speci ed. This descriptor is the smallest le descriptor
which is not currently in use in the calling c actor. The o ag parameter is built
as a bitwise inclusive conjunction (OR) of one of the following ags which de nes
the type of access which will be allowed on the object:
 O RDONLY: the memory object may be accessed only for reading;
 O RDWR: the memory object may be accessed for reading and writing.
The O CREAT and O EXCL ags may also be set. When the O CREAT is set, if
the shared memory object does not exist, it is created and the third parameter
de nes the rights of the users with the same interpretation as for Posix les. If
the O EXCL ag is also set, the operation will fail if the shared memory object
already exists. A failing call returns -1 and errno is set to indicate the error:
errno value error
ENOENT unknown object and O CREAT not set
EACCESS permission denied
EEXIST object exists and O CREAT and O EXCL are set
EINVAL object's name is too long (>SHM PATHMAX) or o ag is incorrect
EMFILE two many le descriptors in the c actor
ENOSPC insucient space
EFAULT name out of the c actor's address space
ENOSYS the function is not supported (feature not active)

7.8.3 Unlinking a shared memory object: shm unlink


Once it has been created, a shared memory object exists until is explicitly
unlinked by calling:
#include <sys/mman.h>
int shm unlink(const char *name);
The actual deletion of the memory object will be postponed until all open and
map references to the shared memory object have been removed.
7.8. The POSIX shared memory 193
If the call succeeds is returned, otherwise -1 is returned and errno is set to
0
, ,
EINVAL ENOENT EFAULT or ENOSYS.
7.8.4 Setting the size of a shared object: ftruncate
Once it has been created and before it may be used, the size of a memory object
must be set by calling the primitive
#include <unistd.h>
int ftruncate(
void *descriptor,
off t length
);
This call sets the size of the memory object referenced by descriptor to length
(descriptor should have been obtained by calling shm open). Upon success, the
call returns 0, otherwise -1 is returned and errno is set to EBADF if descriptor
is not valid or EINVAL if descriptor is not open for writing.
7.8.5 Mapping a shared memory object: mmap
The mapping of a shared memory object in the address space of a c actor is
performed by calling
#include <sys/mman.h>
void *mmap(
void *addr,
size t len,
int prot,
int ags,
int ldes,
off t o
);
In this call:
 ldes is a descriptor identifying the shared memory object and must have
been obtained by a successful call to shm open;
 addr is an address in the calling c actor's address space and len is a number of
bytes which should have been allocated to that address (it is the number of bytes
of the memory object that will be mapped into the c actor's address space). The
value of this parameter will be rounded to the next page-aligned value and any
partial page at the end of an object will be zero- lled automatically;
 o de nes the position in the shared memory object of the rst byte to be
mapped. Its value must be a multiple of the page size;
 prot speci es what kind of access will be allowed. It is built as a combination
of the following ags:
194 Chapter 7. Basic memory management
- PROT READ: data may be read;
- PROT WRITE: data may be written;
- PROT EXECUTE: data may be executed;
 ags provides information about the handling of the mapped data. Its value
is a bit-wise inclusive OR of the following ags:
- MAP PRIVATE: changes are private. The actor will use a copy of the object:
the shared object will be duplicated in the calling c actor's address space
by rgnInitFromActor;
- MAP SHARED: changes are shared. The c actor will use the shared object:
this object will be duplicated in the calling c actor's address space by
rgnMapFromActor. Only one of MAP PRIVATE or MAP SHARED can be set;
- MAP FIXED: the value of addr is mandatory. If this ag is set, addr must
be page-aligned. If it is not set, the value of addr is ignored.
When returning from a successful call, the address of the mapping is returned:
the corresponding range of addresses will be locked in memory. If the call fails,
the MAP FAILED value is returned and the errno variable is set:
errno's value error
ENXIO invalid range for the object
EACCESS incompatible value of prot and the opening mode
EBADF wrong value of ldes
EINVAL ags is invalid: neither MAP PRIVATE nor MAP SHARED is set or
o is not a multiple of the page size or
MAP FIXED is set and addr is not page-aligned
ENOMEM not enough space for locking the object or
MAP FIXED is set and the address range in the c actor's address
space is not valid
EAGAIN the object's size is 0 or not enough room for locking
ENOSYS the function is not supported (feature not active)
7.8.6 Unmapping a shared memory object: munmap
Addresses which have been used for mapping a shared memory object may be
unmapped by calling
#include <sys/mman.h>
void *munmap(
void *addr,
size t len
);
The value of addr must be page-aligned and the value of len will be automatically
rounded to the next page boundary. If the call succeeds 0 is returned, otherwise
-1 otherwise is returned and errno is set to EINVAL or ENOSYS.
Chapter 8
Threads
8.1 Introduction
The c actors we created up until now using the arun utility or by calling one
of the afexec primitives are essentially classic processes like those found in
traditional UNIX systems: a set of resources is provided to a single ow of
control. The main and fundamental di erence is that from the internal point
of view the structure in not monolithic; two abstractions (address space and
ow of execution) are well identi ed and isolated by the kernel. This allows the
creation of other ows of control (threads) in an existing actor which becomes
multi-threaded. A thread is created in an actor and will remain attached to this
actor during its lifetime, this actor is known as the thread's home actor. A
thread will normally execute in the context of its home actor but will sometimes
execute in another one (its execution actor) through local access points which
may be invoked synchronously by calling the lapInvoke primitive (cf. 11.5.2.)
or asynchronously when executing event handlers (chapter 14).
All threads of an actor share the resources of that actor (address space, com-
munication ports, le context, . . . ) but each thread has its own attributes and
characteristics:
 a local identi er in its home actor, meaningful in the context of this actor;
 a context corresponding to the state of the processor (registers, program
counter, stack pointer, privilege, . . . );
 an execution mode or privilege. It may be:
- the user privilege: a thread with that privilege can only access the
address space of its actor and executes in non-privileged mode;
- the supervisor privilege: a thread executing with that privilege can
access the supervisor address space (data and procedures) and will be
allowed to execute privileged processor instructions.
195
196 Chapter 8. Threads
When created, a thread has an initial privilege:
- a thread of a supervisor actor is by de nition created with the supervisor
privilege and will have that privilege during its entire lifetime. It has
unlimited access to the kernel's services;
- a thread of a user actor may be created
 either with the user privilege. This is the most common case. In
that case the thread may temporarily acquire the supervisor privi-
lege either by invoking a system call (some system calls are limited
to supervisor threads and others to user threads belonging to trusted
user actors) or by invoking a local access point of a supervisor actor.
This invocation will be either synchronous by making an explicit
lapInvoke call (see chapter 11) or asynchronous if the lap is con-
nected to an exception or an abort handler (see chapter 14);
 or with the supervisor privilege. Creating a supervisor thread re-
quires that the invoking thread is either running as part of a system
(trusted) actor or is currently executing with supervisor privilege.
 stacks for execution
- a system stack: is automatically allocated in the supervisor address
space when the thread is created. This stack is used for system calls and
when the thread is executing in supervisor mode. As stated in chapter
7 the size of this stack is kern.exec.dflSysStackSize (con guration's
parameter which has 0x3000 as the default value). This stack is the only
stack created for supervisor threads;
- a user stack: when a user thread is created, this stack has to be fully
de ned (address and size) in the address space of its home actor. We
have seen that when a user c actor is created by the AM actor, a stack
is automatically allocated to its main thread. The system does not auto-
matically change the size of that stack when stack over ow occurs. We
will see in chapter 14 how it is possible to change the size of the stack
dynamically by catching the corresponding exception. The user stack of
a thread belongs to the address space of its actor and thus it may be
accessed by any thread of that actor. One should be aware that over ow
of a user thread stack may corrupt adjacent regions and thus corrupt the
contents of the stack of another thread. Therefore, a careful choice of the
start address of this stack and of its size is essential for the correctness of
applications;
8.1. Introduction 197
 a state: determines whether the thread is a candidate for scheduling (it is
active) or temporarily sleeping because it is waiting for an event;
 a scheduling class and a priority: these are used by the scheduler to allocate
the processor(s). Threads are independent entities which are considered by the
kernel for scheduling. Each thread has its own scheduling attributes and if a
thread is blocked on an event, other threads belonging to the same actor are
not blocked and may be scheduled.
Threads may be dynamically created and deleted, a thread of an actor may
create or delete a thread in another one (currently, the actors must belong to
the same site).
The gure given below summarizes these concepts:
ports ports
user actor A1 user actor A2
threads
R1 R5 US3

R2 US1
US2 R6
R3

R4 PC1 R7 PC3

user sctors's user sctors's


address space address space
S supervisor actor A3
U
P SS2
E PC2
R
V R8 SS3
I
S a functionto regions 
O SS4 associated
a lap
R thread's descriptor thread's descriptor
A SS1 Priv: user Priv: supervisor
D supervisor user stack US1 user stack US2
D stacks syst stack SS1 syst stack SS2
R of user prog counter PC1 prog counter PC2
E and PC4
S supervisor thread's descriptor  thread's descriptor
S threads
S Priv: user Priv: supervisor
P user stack US3 user stack:none
orkernel's
syst stack SS3 syst stack SS4
A supervisor
actor's prog counter PC3 prog counter PC4
C code
E
198 Chapter 8. Threads
- A1 is a user actor and is the HOME actor of the threads and , A2 is also a
user actor which holds the thread and nally A3 is a supervisor actor which
contains the  thread. The stacks of and are contiguous: if an over ow
occurs on 's stack, the contents of 's stack will be corrupted;
- Ri are regions either in the user address space of an actor or in the supervisor
address space;
- threads and threads have a user stack in the user address space of actor
A1, has a user stack in the user address space of its HOME actor A2 ( ,
and were created with the user privilege). Thread , which was created as a
supervisor thread (as its HOME actor is the A3 supervisor actor) has no user
stack;
- the four threads have a system stack in the supervisor address space; these
stacks were allocated by the system when the threads were created;
- threads and currently have the USER privilege: they are executing inside
their home actors. The program counters of these threads point to addresses in
their home actor's address space. In contrast, thread , though belonging to the
A1 user actor, is currently executing in supervisor mode. Its program counter
(PC2) is pointing to an address in the supervisor address space: it could have
been promoted, for example, by calling a function of a supervisor actor through
a local access point (lap).
8.2 Threads names and states
8.2.1 Thread identi cation
8.2.1.1 Local identi cation
A thread can refer to itself by using the K MYSELF symbolic constant. The
K MYACTOR constant will always refer to the execution actor of the current
thread.
As we said, a thread can be locally named in its actor by a local identi er which
can be retrieved by calling the primitive
KnThreadLid threadSelf(void);
When a thread has to be named in the context of an actor other than its home
actor, the capability of this home actor will be used as a complement to the
local identi er, as in the next call presented below.
8.2.1.2 Symbolic name
A symbolic name (whose length is at most K THREADNAMEMAX, with a default of
16) may be bound to a thread. A call to the threadName primitive allows to
8.2. Threads names and states 199
set or to retrieve the symbolic name of a given thread:
#include <exec/chExec.h>
int threadName(
KnCap *actorCap,
KnThreadLid threadLi,
VmAddr oldName,
VmAddr newName
);
If oldName is NULL the old value is not retrieved and if newName is NULL,
the current symbolic name of the thread is not changed. When successful,
the function returns K OK, otherwise K EINVAL or K EUNKNOWN if actorCap or
threadLi are not valid, or K EFAULT if an address is outside the address space.
8.2.2 Thread states
From the user point of view, a thread may be in one of the following four fun-
damental states:
 ACTIVE: the thread is running or is ready to run (queued in a ready list);
 INACTIVE: the thread has been created in this state. It will become active
after an explicit call to threadActivate. This state is used by applications
which need to control all the initial context (corresponding to the context of
hardware registers) of a thread at creation time before it executes any instruc-
tion (see threadCreate in 8.5.1.1 and threadContext in 8.11);
 STOPPED: the thread has been stopped by calling threadStop and it will
become active again after a threadStart call has been applied;
 WAITING: the thread is currently waiting for an event. This type of
situation occurs when the thread is blocked by calling a primitive such as
threadDelay, ipcCall, mutexGet, . . .
The next gure illustrates these di erent possibilities:
blocking call
INACTIVE WAITING
threadActivate wakeup
threadCreate

threadStop STOPPED

ACTIVE threadStart

We will see later that di erent orthogonal states are possible.


200 Chapter 8. Threads
8.3 Threads scheduling
8.3.1 Overview
Every kernel instance includes a scheduling module which provides one or more
scheduling policies which consist of rules, procedures and criteria used to take
decisions to allocate the processor. These decisions apply to threads: every
thread in the system has a system priority and a scheduling class which de nes
the rules that are applied.
The system priority of a thread belongs in the range [0:255] where 0 (sym-
bolic constant K PRIOMAX) is the highest priority and 255 (symbolic constant
K PRIOMIN) the lowest one.
The default scheduling of the executive is based on a FIFO policy: threads
belong to the same FIFO class of scheduling (called SCHED FIFO).
The ROUND ROBIN con gurable feature allows di erent scheduling policies to be
selected, the Round Robin scheduling policy (this de nes the SCHED RR class)
and the real-time scheduling policy (this de nes the SCHED RT class). When this
feature is enabled, the di erent scheduling classes coexist and share the same
range of system priorities: a given thread belongs to one of these classes.
The following gure illustrates this:

FIFO RR
0

RT
95

155
195

255
default scheduler
with ROUND ROBIN feature

8.3.2 The scheduling classes


As said earlier, the executive core implements a FIFO scheduler and the ROUND ROBIN
feature enables other scheduling classes.
8.3. Threads scheduling 201
8.3.2.1 The FIFO class
This default scheduling policy (designed by the symbolic K SCHED FIFO con-
stant) is a pure priority-based, preemptive fo policy whose main characteristics
are the following:
 every thread has a xed priority which may vary between the two symbolic
K FIFO PRIOMAX (0) and K FIFO PRIOMIN (255) values;
 a thread may only be preempted by another one with a strictly higher prior-
ity. When it is preempted, a thread is placed at the head of the priority queue
for its priority. This means it will be elected when the preempting thread has
completed or blocks (if no thread with a higher priority is ready to run);
 when a thread blocks as a result of a system call, it releases the processor
and so allows the scheduling of another thread; the rst ready to run thread
with the highest priority will be elected. When a thread which was previously
blocked becomes ready to run, it is inserted at the end of the priority queue for
its priority.
The scheduling parameters of a thread belonging to that class are packed into
a data structure of the KnFifoThParms type de ned as
#include <sched/chFifo.h>
typedef struct KnFifoThParms {
KnSchedClass fifoClass; /* it must be K_SCHED_FIFO */
KnFifoPriority fifoPriority; /* in [K_FIFO_PRIOMAX : K_FIFO_PRIOMIN] */
} KnFifoThParms;

8.3.2.2 The RR class


This is designed by the symbolic K SCHED RR constant and is only available
within the ROUND ROBIN feature. It is important to note that this policy coex-
ists with the default FIFO policy.
It is a priority-based preemptive policy with round-robin time slicing. Its main
characteristics are:
 a thread has a xed priority which may vary between the two symbolic
K RR PRIOMAX (0) and K RR PRIOMIN (255) values;
 as for the FIFO class, a thread may be preempted by another one with a
strictly higher priority in which case it is placed at the head of the relevant pri-
ority queue, it will be elected when the preempting thread completes or blocks;
 when it is elected, a thread is given a xed time quantum. If it is still running
when this time expires, it is preempted and inserted at the end of the ready
queue according to its priority level; thus, it yields the cpu to the next thread
ready to run at the same priority level;
202 Chapter 8. Threads
 the time quantum is reset when a thread becomes ready after having blocked.
On the other hand, after having been preempted by another thread with a
strictly higher priority, this time quantum is not reset; thus, the thread will
only run for the remaining time.
The scheduling parameters of threads belonging to that scheduling class are
packed into the following KnRrThParms structure:
#include <sched/chRr.h>
typedef struct KnRrThParms {
KnSchedClass rrClass; /* it must be K_SCHED_RR */
KnRrPriority rrPriority; /* in [K_RR_PRIOMIN : K_RR_PRIOMAX] */
} KnRrThParms;
8.3.2.3 The RT class
This is designed by the symbolic K SCHED RT constant: it is only available within
the ROUND ROBIN feature and coexists with the other policies ( fo and round
robin). It implements the same scheduling policy as the real-time scheduler of
UNIX-SVR4 systems.
The scheduling parameters corresponding to this policy are de ned by the fol-
lowing structure:
#include <sched/chRt.h>
typedef struct KnRtThParms {
KnSchedClass rtClass; /* must be K_SCHED_RT */
KnRtParms *rtParms; /* attributes */
} KnRtThParms;

where the KnRtParms is de ned as:


#include <sched/chRt.h>
typedef struct KnRtParms {
KnRtPriority rtPriority; /* priority */
unsigned long rtQSecs; /* number of seconds */
long rtQNSecs; /* number of nanoseconds */
} KnRtParms;

This policy is basically a round-robin policy with a per-thread de ned time


quantum. The K RT PRIOMIN and K RT PRIOMAX values correspond to the integer
values 100 (the lowest priority) and 159 (the highest priority). The system
priorities corresponding to these priorities are in the range [95:155].
Some prede ned values may be used for the rtQSecs eld when manipulating
the scheduling attributes of a thread:
 RT DEFAULT: default prede ned value;
 RT INFINITE: in nite value;
 RT SAMEQUANTUM: same value as before (or in nite value if the thread is
entering the RT class).
8.3. Threads scheduling 203
8.3.3 Getting or setting scheduling information
8.3.3.1 The default KnThreadDefaultSched class
As seen in the previous sections, a speci c type is associated with every schedul-
ing class. If the ROUND ROBIN feature is enabled, it is convenient to use the
KnThreadDefaultSched generic type whenever the scheduling class of a given
thread is not known. This type is de ned as
#include <sched/chSched.h>
typedef struct KnThreadDefaultSched {
KnSchedClass tdClass;
KnThreadPriority tdPriority;
} KnThreadDefaultSched;
where the tdClass eld has one of the following values:
 K SCHED FIFO or K SCHED DEFAULT choosing the FIFO scheduling policy;
 K SCHED RR enabling the RR scheduling policy.
8.3.3.2 The threadScheduler primitive
The threadScheduler function, described below, allows the scheduling at-
tributes of a given thread to be retrieved and/or to modi ed:
#include <exec/chExec.h>
#include <sched/chSched.h>
int threadScheduler(
KnCap *actorCap,
KnThreadLid threadLi,
void *oldParam,
void *newParam
);
The oldParam and newParam arguments point to a structure of attributes com-
patible with the scheduling class of the thread or to a KnThreadDefaultSched
object. A NULL value for one of these parameters obscures the getting/setting
scheduling attributes operation.
Upon success, the function returns K OK, otherwise a negative error code is
returned:
return value error type
K EINVAL actorCap is an inconsistent capability, or
threadLi is an invalid thread local identi cation
K EUNKNOWN actorCap does not correspond to a reachable actor
K EINVAL structure pointed to by newParam contains invalid values
K ENOMEM system is out of resources
K ENOTIMP requested scheduling policy is not supported
K EFAULT some data points outside the address space of the actor
204 Chapter 8. Threads
8.3.4 Examples
8.3.4.1 Getting scheduling attributes
In the following example:
 if no argument is given when calling the command, the scheduling attributes
of the main thread of the c actor that is dynamically created for executing the
command are printed;
 if the command has at least ve arguments, the rst four are interpreted
as a capability and the others as local identi ers of thread belonging to the
corresponding actor and the attributes of all these threads are printed:
--> cat $CHORUS/sources/threads/prioAttributes/prioAttributes.c
#include "utilities.h"
#include <sched/chSched.h>
#include <sched/chFifo.h>
KnThreadDefaultSched prioParams;
int result, ind;
KnCap capa;
KnThreadLid threadLi;

main(int argc, char *argv[ ]) {


/* less than 5 arguments: current actor and thread are considered */
if(argc <= 5) {
result = threadScheduler(K_MYACTOR, K_MYSELF, &prioParams, NULL);
switch(prioParams.tdClass) {
case K_SCHED_FIFO: printf("Main thread in FIFO class\n");
break;
case K_SCHED_RR : printf("Main thread in RR class\n");
break;
default : printf("Main thread in unknown sched. class\n");
}
printf("Main thread's priority: %d\n", prioParams.tdPriority);
return;
}
/* if at least 5 arguments, decode and print the capability */
readCap(argv + 1, &capa);
fprintCap(stdout, "in actor", &capa);
/* print attributes of given threads in the actor */
for(ind = 5; ind < argc; ind ++) {
threadLi = atoi (argv[ind]);
printf(" thread %d: ", threadLi);
result = threadScheduler(&capa, threadLi, &prioParams, NULL);
if(result < 0)
printf(" unknown scheduling attributes\n");
8.3. Threads scheduling 205
else {
switch(prioParams.tdClass) {
case K_SCHED_FIFO: printf("FIFO, ");
break;
case K_SCHED_RR : printf("RR, ");
break;
default : printf("??, "); }
printf("%d\n", prioParams.tdPriority);
}
}
}
In the next sequence, we rst use the command without argument to get the
attributes of the main thread of the actor dynamically created when executing
the application (as said before, this thread belongs to the fo scheduling class
and its priority is 140):
--> neon prioAttributes_u
started aid = 2
Main thread in FIFO class
Main thread's priority: 140
We then use the command to retrieve the scheduling attributes of the various
threads of the kern and C INIT system actors:
--> rsh neon arun cs -la 1
started aid = 2
ChorusOS r4.0.0 Site 0 Time 16h 54m 20
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000001 869da80a 00000001 00000001 0001 SUP STARTED 003 kern
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0003 000 00000010 00000010 ff6a50 0- 1- 0 dkiThread
0002 094 00000000 00000000 ff6c04 0- 1- 0 IpcPortHandler
0001 256 60831850 60831850 ff6db8 0- 1- 0 Idle
..........................
--> rsh neon arun cs -la 21
started aid = 2
ChorusOS r4.0.0 Site 0 Time 16h 55m 53
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000015 869da80a 00000015 00000001 0021 SUP STARTED 001 C_INIT
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0004 030 00000060 00000000 ff689c 0- 1- 0 rshd_handler
..........................
--> neon prioAttributes_u 20000001 869da80a 1 1 3 2 1
started aid = 2
in actor: 20000001 869da80a 1 1
thread 3: FIFO, 0
thread 2: FIFO, 94
thread 1: FIFO, 256
206 Chapter 8. Threads
--> neon prioAttributes_u 20000015 869da80a 15 1 3 4
started aid = 2
in actor: 20000015 869da80a 15 1
thread 3: unknown scheduling attributes (= unknown thread
thread 4: FIFO, 30

8.3.4.2 Modifying scheduling attributes


In this example, the main thread of the actor requests to enter the round robin
scheduling class. If the command is called with an argument, its value is used
as the new priority of the thread, otherwise the priority is not changed:
--> cat $CHORUS/sources/threads/modifyPrio/modifyPrio.c
#include <stdio.h>
#include <chorus.h>
#include <sched/chSched.h>
#include <sched/chRr.h>
KnThreadDefaultSched prioParams;
main(int argc, char *argv[ ]) {
int result;
KnCap capa;
KnThreadLid threadLi;
actorSelf(&capa);
fprintCap(stdout, "New actor", &capa);
printf("Main thread: %d\n", threadSelf( ));
/* get value of scheduling attributes */
if (argc == 1)
threadScheduler(K_MYACTOR, K_MYSELF, &prioParams, NULL);
else
prioParams.tdPriority = atoi(argv[1]);
prioParams.tdClass = K_SCHED_RR;
result = threadScheduler(K_MYACTOR, K_MYSELF, NULL, &prioParams);
if (result != K_OK)
fprintf(stderr, "threadScheduler: %s\n", strSysError(result));
threadDelay(K_NOTIMEOUT);
}
--> neon-n modifyPrio_u 200 &
[1] 13234
--> started aid = 2
New actor: 20000038 869da80a 2 174
Main thread: 9
--> neon prioAttributes_u 20000038 869da80a 2 174 9
started aid = 22
in actor: 20000038 869da80a 2 174
thread 9: RR, 200
--> rsh neon akill 2
-->
8.3. Threads scheduling 207
8.3.4.3 Fifo scheduling vs Round Robin scheduling
In the next application, if the command is executed without parameters, the
main thread executes within the default scheduling attributes ( fo scheduling
class and priority 140). If the command is called with parameters, it is executed
in the round robin scheduling class with that default priority. The code is
composed of two nested loops: the internal loop simulates a long computation
which is performed 3 times (number of steps of the external loop):
--> cat $CHORUS/sources/threads/fifo_rr/fifo_rr.c
#include <stdio.h>
#include <chorus.h>
#include <sched/chSched.h>
#include <sched/chFifo.h>
#include <sched/chRr.h>
KnThreadDefaultSched prioParams;
int externLoop, internLoop;
main(int argc, char *argv[ ]) {
threadScheduler(K_MYACTOR, K_MYSELF, &prioParams, NULL);
if (argc != 1) {
prioParams.tdClass = K_SCHED_RR;
threadScheduler(K_MYACTOR, K_MYSELF, NULL, &prioParams);
}
for (externLoop = 0; externLoop < 3; externLoop ++) {
for (internLoop = 0; internLoop < 10000000; internLoop ++)
; /* empty long loop */
printf("actor %d : %d\n", agetId( ), externLoop);
}
}
-->
We rst concurrently request the execution of three samples of the application
in the fo scheduling class. The three actors are created as the thread of the
C INIT actor has a higher priority, and the main threads of the c actors execute
in the order in which the actors were created:
--> neon-n fifo_rr_u & neon-n fifo_rr_u & neon-n fifo_rr_u&
[1] 13408
[2] 13409
[3] 13411
--> started aid = 22
started aid = 2
started aid = 23
actor 22 : 0
actor 22 : 1
actor 22 : 2
208 Chapter 8. Threads
actor 2 : 0
actor 2 : 1
actor 2 : 2
actor 23 : 0
actor 23 : 1
actor 23 : 2

We now do the same using the round robin scheduling; the three actors are
created and we can observe that an interleaving of the messages from the three
threads which re ects the round robin policy
--> neon-n fifo_rr_u 1 & neon-n fifo_rr_u 1& neon-n fifo_rr_u 1&
[1] 13415
[2] 13416
[3] 13418
--> started aid = 23
started aid = 2
started aid = 22
actor 23 : 0
actor 2 : 0
actor 23 : 1
actor 2 : 1
actor 22 : 0
actor 22 : 1
actor 2 : 2
actor 23 : 2
actor 2 : 2
actor 22 : 2

8.4 Delaying a thread


8.4.1 The threadDelay primitive
In several previous examples of this book, we delayed the execution of the
current thread either for a nite duration de ned as a KnTimeVal value, or for
one similar to the K NOTIMEOUT constant by calling the primitive
#include <exec/chExec.h>
int threadDelay(KnTimeVal *delay);
This type of call puts the calling thread to sleep for the duration speci ed by
delay:
 the K NOTIMEOUT and K NOTIMEOUT NOABORT values specify an in nite delay;
the rst one for an abortable call of threadDelay and the second one for a
non-abortable one (we will present the di erence in 8.8);
 the K NOBLOCK value (or 0) speci es a null delay. The calling thread is still
ready to run after the call; it is placed at the end of the running queue relative
8.4. Delaying a thread 209
to its priority and yields the processor to another ready thread;
 a call to the K MILLI TO TIMEVAL macro allows a value expressed in milli-
seconds (the second parameter) to be changed to a KnTimeVal type, and to
write this value to the address pointed to by the rst argument.
The threadDelay function always returns a negative error value:
- K EINVAL if *delay is not a valid KnTimeVal structure;
- K ETIMEOUT if the timeout occurred;
- K EABORT if the thread was aborted while delayed (see 8.8).
For example to impose a delay of 4.5 seconds on a thread, the following sequence
would be used:
KnTimeVal delay;
..........
K_MILLI_TO_TIMEVAL(&delay, 4500); /* delay in milli-seconds */
threadDelay(&delay);
..........

8.4.2 Example
This example illustrates the use of the K NOBLOCK argument in a threadDelay
call to yield the processor to other threads with the same priority. The main
thread executes two successive loops; the number of steps of these loops are
de ned by the value of the rst parameter of the command. Furthermore, if the
command is executed with a second argument, it calls threadDelay(K NOBLOCK)
before entering the second loop:
--> cat $CHORUS/sources/threads/procYield/procYield.c
#include <stdio.h>
#include <chorus.h>
main(int argc, char *argv[ ]) {
int ind, loop;
loop = atoi(argv[1]); /* decode argv[1] as an integer */
fprintf(stderr, "c_actor %d before the first loop\n", agetId( ));
for(ind = 0; ind < loop; ind ++)
; /* first empty loop */
fprintf(stderr, "c_actor %d after the first loop\n", agetId( ));
if (argc > 2)
threadDelay(K_NOBLOCK)
for(ind = 0; ind < loop; ind ++);
; /* second empty loop */
fprintf(stderr, "c_actor %d after the second loop\n", agetId( ));
}
We rst execute three samples of the command for a small value of loop without
calling threadDelay between the loops. We observe that once an actor has been
210 Chapter 8. Threads
created, its main thread may execute completely before C INIT receives the
request to create a new actor. The three actors have the same local identi er:
--> neon-n procYield_u 100000 & neon-n procYield_u 100000 &
neon-n procYield_u 100000 &
[1] 13564
[2] 13565
[3] 13567
--> started aid = 22
c_actor 22 before the first loop
c_actor 22 after the first loop
c_actor 22 after the second loop
started aid = 22
c_actor 22 before the first loop
c_actor 22 after the first loop
c_actor 22 after the second loop
started aid = 22
c_actor 22 before the first loop
c_actor 22 after the first loop
c_actor 22 after the second loop

The situation is di erent if loop has a large value: we observe the preemption
of the thread of the rst created actor by the thread of the C INIT actor when
it receives the requests to create new actors. Once they are all created, the
three threads of the new actors execute sequentially:
--> neon-n procYield_u 10000000 & neon-n procYield_u 10000000 &
neon-n procYield_u 10000000 &
[1] 13570
[2] 13571
[3] 13573
--> started aid = 22 (= rst actor is created
c_actor 22 before the first loop (= thread starts execution
started aid = 2 (= second actor is created
c_actor 22 after the first loop (= rst thread resumes
started aid = 23 (= third actor is created
c_actor 22 after the second loop (= rst thread terminates
c_actor 2 before the first loop (= ^ main thread
c_actor 2 after the first loop (= j of the second
c_actor 2 after the second loop (= _ actor
c_actor 23 before the first loop (= ^ main thread
c_actor 23 after the first loop (= j of the third
c_actor 23 after the second loop (= _ actor
If the commands are launched with two arguments, the thread calls threadDelay.
Thus, the executing thread releases the processor and is placed at the end of
the corresponding running queue.
8.5. Creating and deleting a thread 211
--> neon-n procYield_u 50000000 1 & neon-n procYield_u 50000000 1 &
neon-n procYield_u 50000000 &
[1] 13600
[2] 13601
[3] 13603
--> started aid = 23
c_actor 23 before the first loop
started aid = 2
started aid = 22
c_actor 23 after the first loop (= rst thread yields processor
c_actor 2 before the first loop (=second thread executes
c_actor 2 after the first loop (=second thread yields processor
c_actor 22 before the first loop (=third thread executes
c_actor 22 after the first loop (=third thread yields processor
c_actor 23 after the second loop (= rst thread resumes execution
c_actor 2 after the second loop (=second thread resumes execution
c_actor 22 after the second loop (=third thread resumes execution

8.5 Creating and deleting a thread


8.5.1 Creating a thread
8.5.1.1 The threadCreate primitive
Thread creation is provided as a standard service by the core executive by
calling the primitive
#include <exec/chExec.h>
int threadCreate(
KnCap *actorCap,
KnThreadLid *threadLi,
KnThreadStatus status,
void *schedParam,
void *startInfo
);

where:
 the actorCap argument designates the actor where the thread is to be created.
The K MYACTOR value indicates the current actor;
 the status argument may have the value K ACTIVE or K INACTIVE;
 the structure pointed to by the schedParam argument de nes the attributes
for scheduling the new thread. If its value is NULL, the new thread inherits the
scheduling characteristics (scheduling class and priority) of the calling thread;
212 Chapter 8. Threads
 the structure pointed to by the startInfo argument de nes the initial state of
the new thread. It is machine dependent and corresponds, on most conventional
processors, to the KnDefaultStartInfo f type containing the following elds:
- dsType whose type is KnStartInfoType: K DEFAULT START INFO is a stan-
dard prede ned value which will generally be used;
- dsSystemStackSize which is an unsigned int: it gives the size of the
system stack expected for the thread; the K DEFAULT STACK SIZE prede-
ned value is generally used;
- dsEntry whose type is KnPc: it gives the entry point of the thread. It
is generally de ned as the address of a function which will be executed
when the thread starts;
- dsUserStackPointer which points to a void: for a user thread it gives
the initial user stack pointer. The user stack may be allocated by calling
the malloc function or by creating a new region in the actor. The top
of the stack is generally accessed through a dedicated register. On most
processors, (ix86 or Sparc for instance), allocation on this stack is per-
formed towards lower addresses. This is because the value of the stack
register decreases when a PUSH instruction is performed and increases for
a POP instruction. Thus, before calling the threadCreate primitive, the
value of the dsUserStackPointer eld of the structure has to be set to
the end of the space which has been allocated for the thread user stack. A
last remark concerns the de nition of the location pointed to by the stack
pointer. It is generally the last word allocated on the stack: this means
that the PUSH operation rst decrements the value of the stack pointer
and allocates the word corresponding to the resulting value. Thus, if the
stack has been allocated using p=malloc(n), the initial value of the eld
may be p + n (that is the case on ix86 processors) or any value p + n - i
word-aligned. In our examples we will use p + n - sizeof(long) as the
initial value.
If the thread is created with supervisor privilege, it will use its system
stack which is automatically allocated by the system during thread cre-
ation and this eld is not used;
- dsPrivilege whose type is KnThreadPrivilege: it has as a value of
either K USERTHREAD if the thread is a standard thread or K SUPTHREAD if
the new thread is a supervisor thread.
If the call succeeds (and a new thread is created), K OK is returned and the
local identi er of the new thread is returned at the location pointed to by the
threadLi argument.
8.5. Creating and deleting a thread 213
If an error occurs, a negative error code is returned:
value of errno error type
K EINVAL actorCap is an inconsistent capability
K EUNKNOWN actorCap does not correspond to a reachable actor
K EFAULT some data points outside the address space of the actor
K EPRIV the calling thread is not allowed to create a supervisor thread
K ENOTIMP the scheduling parameter requires a non-supported policy

8.5.1.2 Examples
a) We rst develop two functions which allow us to create threads dynamically.
These functions will be added to the utilities library. We add their proto-
types and some speci c chorus header les in the "utilities.h" header le
associated to that library:
--> cat $CHORUS/include/utilities.h
#include <stdio.h>
#include <chorus.h>
#include <sched/chSched.h>
#include <sched/chFifo.h>
#include <sched/chRr.h>
void fprintCap(FILE *, char *, KnCap *);
void readCap(char **, KnCap *);
KnThreadLid newThread(KnPc function, int status, char *symbName);
KnThreadLid newThreadAttr(
KnCap *actorCap, KnPc function, int status,
int stackSize, int class, int prio);
The newThread function creates a new thread in the current actor:
 function is the entry point of the new thread;
 status has the value (K ACTIVE or K INACTIVE) de ning the initial state of the
new thread;
 symbName is the symbolic name of the new thread. A NULL value may be
used if the new thread has no name.
The other attributes of the new thread are chosen as follows:
 the privilege of the new thread is deduced from the privilege of the actor
where it is created. It is a supervisor thread if the actor is a supervisor actor
and a user thread otherwise;
 its scheduling attributes are the same as the attributes of the calling thread;
 if the thread has the user privilege a user stack is dynamically allocated in
the actor by calling the malloc function. The size of the stack is set by the
USER STACK SIZE constant, prede ned as four pages in the source.
214 Chapter 8. Threads
--> cat $CHORUS/sources/utilities/lib/newThread.c
#include "utilities.h"
#define USER_STACK_SIZE 0x4000
KnThreadLid newThread(KnPc entryPoint, int status, char *symbName) {
KnActorPrivilege actorPriv;
KnDefaultStartInfo_f startInfo;
char *userStack;
KnThreadLid childLident = -1;
int result;
/* set default startInfo fields */
startInfo.dsType = K_DEFAULT_START_INFO;
startInfo.dsSystemStackSize = K_DEFAULT_STACK_SIZE;
/* get actor's privilege and set thread privilege */
result = actorPrivilege(K_MYACTOR, &actorPriv, NULL);
if(result != K_OK) {
fprintf(stderr, "error on actorPrivilege: %s\n", strSysError(result));
return -1;
}
startInfo.dsPrivilege =
(actorPriv == K_SUPACTOR) ? K_SUPTHREAD : K_USERTHREAD;
/* allocation of the thread's user stack for a user thread */
if (startInfo.dsPrivilege == K_USERTHREAD) {
if((userStack = (char *) malloc(USER_STACK_SIZE)) == NULL) {
fprintf(stderr, "error in stack allocation\n");
return -1;
}
startInfo.dsUserStackPointer =
(char *) userStack + USER_STACK_SIZE - sizeof(long);
}
else
startInfo.dsUserStackPointer = NULL;
/* set entry point for the new thread */
startInfo.dsEntry = entryPoint;
/* create new thread in status state */
result = threadCreate(K_MYACTOR, &childLident, status, NULL, &startInfo);
if (result == K_OK) {
if(symbName != NULL)
threadName(K_MYACTOR, childLident, NULL, symbName);
return childLident;
}
else {
fprintf(stderr, "error on threadCreate: %d\n", strSysError(result));
if (startInfo.dsPrivilege == K_USERTHREAD)
free(userStack);
return -1;
}
}
8.5. Creating and deleting a thread 215
The newThreadAttr function is a modi ed version of the previous one. It cre-
ates a new thread:
 whose home actor's capability is passed as rst argument when calling the
function;
 that has explicit attributes (user stack's size, scheduling class and priority).
Since the thread may be created in an actor di erent from the current actor, the
stack cannot be allocated using a malloc call. A new region is created in the
target actor by calling rgnAllocate, except if it is a supervisor thread or if the
size is 0. In this case, it may be useful to create an inactive user thread whose
user stack is not de ned immediately, but is created later by copying the stack
of another thread (it is the case in the example given in 8.12 of a simulation of
the Unix fork primitive).
Furthermore, if the thread is created in an actor di erent from the current one,
we will assume that the entry point of the new thread is passed as a virtual
address matching a function in the target actor's virtual address space (and not
in the address space of the current one).
--> cat $CHORUS/sources/utilities/lib/newThreadAttr.c
#include "utilities.h"

KnThreadLid newThreadAttr(
KnCap *actorCap, /* target actor's capability */
KnPc entryPoint, /* new thread's entry point */
int status, /* status: K_ACTIVE or K_INACTIVE */
int stackSize, /* user stack's size in bytes */
int prioClass, /* scheduling class: K_SCHED_FIFO or K_SCHED_RR */
int prio /* thread's priority */
) {

KnRgnDesc rgnDesc; /* for the user stack of the new thread */


KnDefaultStartInfo_f startInfo;
KnThreadLid childLident = -1;
KnActorPrivilege actorPriv;
KnThreadDefaultSched schedParams; /* scheduling attributes */
long *userStack; /* for the user stack */
int result;

/* setting default startInfo fields */


startInfo.dsType = K_DEFAULT_START_INFO;
startInfo.dsSystemStackSize = K_DEFAULT_STACK_SIZE;
/* setting the thread's privilege */
result = actorPrivilege(actorCap, &actorPriv, NULL);
if(result != K_OK)
return result;
216 Chapter 8. Threads
startInfo.dsPrivilege =
(actorPriv == K_SUPACTOR) ? K_SUPTHREAD : K_USERTHREAD;
/* setting entry point for the new thread */
startInfo.dsEntry = entryPoint;
/* setting scheduling attributes of the new thread */
switch(prioClass) {
case K_SCHED_DEFAULT :
case K_SCHED_FIFO :
case K_SCHED_RR : schedParams.tdClass = prioClass;
schedParams.tdPriority = prio;
break;
default : return -1;
}
/* allocate a region for user's stack if necessary */
startInfo.dsUserStackPointer = NULL;
if(startInfo.dsPrivilege == K_USERTHREAD && stackSize > 0) {
rgnDesc.startAddr = 0x0; /* K_ANYWHERE will be used */
rgnDesc.size = stackSize; /* stack's size */
rgnDesc.options = K_ANYWHERE | K_WRITABLE | K_FILLZERO | K_NODEMAND;
result = rgnAllocate(actorCap, &rgnDesc);
if(result != K_OK)
return result;
startInfo.dsUserStackPointer =
(char *) rgnDesc.startAddr + stackSize - sizeof(long);
}
result = threadCreate(actorCap, &childLident, status,
&schedParams, &startInfo);
if (result == K_OK)
return childLident;
else {
rgnDesc.options = 0;
rgnFree(actorCap, &rgnDesc);
fprintf(stderr, "error on threadCreate: %d\n", strSysError(result));
return -1;
}
}

b) Creating threads in the current actor.


We rst use the newThread function to create two new threads in the current
c actor. After having displayed a rst message, the threads enter an active loop
and then display a new message before entering an in nite delay. The new
threads have the same scheduling attributes as the calling thread (the main
thread of the c actor).
8.5. Creating and deleting a thread 217
--> cat $CHORUS/sources/threads/thread1/thread1.c
#include "utilities.h"
#include <sched/chSched.h>

int infiniteSleep( ) {
int ind;
printf(" I am the new thread %d\n", threadSelf( ));
/* long empty loop */
for(ind = 0; ind < 50000000; ind ++)
; /* long empty loop */
printf(" %d ==> I'm going to sleep\n", threadSelf( ));
/* entering an infinite delay */
threadDelay(K_NOTIMEOUT);
}

main(int argc, char *argv[ ]) {


KnThreadLid threadLi1, threadLi2;
KnThreadDefaultSched schedParams;
KnCap actorCap;
int ind;
actorSelf(&actorCap);
fprintCap(stdout, "actor", &actorCap);
if (argc != 1) {
schedParams.tdClass = K_SCHED_RR;
schedParams.tdPriority = 150;
threadScheduler(K_MYACTOR, K_MYSELF, NULL, &schedParams);
}
threadLi1 = newThread((KnPc) infiniteSleep, K_ACTIVE, "thread1");
printf("A new thread has been created. Its local identifier is %d\n",
threadLi1);
for(ind = 0; ind < 30000000; ind ++);
threadLi2 = newThread((KnPc) infiniteSleep, K_ACTIVE, "thread2");
printf("A new thread has been created. Its local identifier is %d\n",
threadLi2);
for(ind = 0; ind < 30000000; ind ++);
printf("Main thread (%d): I'm going to sleep\n", threadSelf( ));
threadDelay(K_NOTIMEOUT);
}
-->
In the rst execution, the main thread is executed in the fo scheduling class.
Therefore, once it starts, it keeps the processor until it goes to sleep (it could
be preempted by a thread with a higher priority but in that case, it will be
put at the head of its waiting queue). We can observe from the results that,
due to that scheduling policy, the main thread executes until it enters its call
to threadDelay, then the rst newly created thread executes until it enters its
call to threadDelay and nally the second newly created thread executes:
218 Chapter 8. Threads
-1-> neon thread1_u
started aid = 22
actor: 20000083 869da80a 16 0
A new thread has been created. Its local identifier is 7
A new thread has been created. Its local identifier is 8
Main thread (9): I'm going to sleep
I am the new thread 7
7 ==> I'm going to sleep
I am the new thread 8
8 ==> I'm going to sleep
We can check that the actor and the three threads exist by using the application
prioAttributes that we developed earlier:
-2-> neon prioAttributes_u 20000083 869da80a 16 0 7 8 9
started aid = 2
in actor: 20000083 869da80a 16 0
thread 7: FIFO, 140
thread 8: FIFO, 140
thread 9: FIFO, 140
-2-> rsh neon akill 22
-2->
-1->
We now execute the same application with a main thread belonging to the round
robin scheduling class. The annex threads inherit the scheduling attributes of
the main thread and we can observe the round robin behavior of the three
threads of our actor from the following results:
-1-> neon thread1_u 1
neon thread1_u 1
started aid = 22
actor: 20000087 869da80a 16 1
A new thread has been created. Its local identifier is 7
I am the new thread 7
A new thread has been created. Its local identifier is 8
I am the new thread 8
7 ==> I'm going to sleep
Main thread (9): I'm going to sleep
8 ==> I'm going to sleep
-2-> neon prioAttributes_u 20000087 869da80a 16 1 7 8 9
started aid = 2
in actor: 20000087 869da80a 16 1
thread 7: RR, 150
thread 8: RR, 150
thread 9: RR, 150
-2-> rsh neon akill 22
-2->
-1->
8.5. Creating and deleting a thread 219
We have modi ed the source given above to get a new thread2 application. The
newThreadAttr function is used instead of newThread to create new threads in
the actor with di erent scheduling classes and di erent priorities:
...............
threadLi1 = newThreadAttr(K_MYACTOR, (KnPc) infiniteSleep, K_ACTIVE,
4096, K_SCHED_RR, 140);
...............
threadLi2 = newThreadAttr(K_MYACTOR, (KnPc) infiniteSleep, K_ACTIVE,
4096, K_SCHED_FIFO, 130);
...............
We rst execute the application without changing the scheduling class of the
main thread; thus, this thread executes until a thread with a higher priority is
ready. The second created thread has a higher priority and it is immediately
scheduled after it has been created. When it enters the sleeping state, the main
thread continues and when it enters the sleeping state, the rst created thread
can execute:
-1-> neon thread2_u
started aid = 22
actor: 20000089 869da80a 16 4cf
A new thread has been created. Its local identifier is 7
I am the new thread 8
8 ==> I'm going to sleep
A new thread has been created. Its local identifier is 8
Main thread (9): I'm going to sleep
I am the new thread 7
7 ==> I'm going to sleep

-2-> neon prioAttributes_u 20000089 869da80a 16 4cf 7 8 9


started aid = 2
in actor: 20000089 869da80a 16 4cf
thread 7: RR, 140
thread 8: FIFO, 130
thread 9: FIFO, 140
-2-> rsh neon akill 22
-2->
-1->
Finally, we execute the command with a main thread in the round robin schedul-
ing class. We can observe an initial round robin between the main thread and
the rst created new thread since they have the same priority and belong to the
RR scheduling class. But, once the second thread has been created, it gains the
processor since it owns a higher priority and keeps it until it enters its sleeping
state (because it belongs to the fo scheduling class):
220 Chapter 8. Threads
-1-> neon thread2_u 1
started aid = 22
actor: 2000009f 869da80a 16 0
A new thread has been created. Its local identifier is 7
I am the new thread 7
I am the new thread 8
8 ==> I'm going to sleep
A new thread has been created. Its local identifier is 8
7 ==> I'm going to sleep
Main thread (9): I'm going to sleep
-2-> neon prioAttributes_u 2000009f 869da80a 16 0 7 8 9
started aid = 2
in actor: 2000009f 869da80a 16 0
thread 7: RR, 140
thread 8: FIFO, 130
thread 9: RR, 140
-2->
-1->

c) Creating a thread in an actor di erent from the current actor


We now use the newThreadAttr function with all its exibility to create a thread
in an actor di erent from the current actor.
First, we create an actor for the targetActor application: it contains the de -
nition of a function intended to be executed by a thread which will be created
externally. Before entering an in nite delay in the main thread, the address of
the function is printed:
-2-> cat $CHORUS/sources/threads/targetActor/targetActor.c
#include "utilities.h"
KnCap actorCap;
void annexThread( ) {
actorSelf(&actorCap);
fprintCap(stdout,
" +++ Entering annexThread function in target actor",
&actorCap);
printf(" +++ Executing thread %d", threadSelf( ));
fprintCap(stdout, " in actor ", &actorCap);
threadDelay(K_NOTIMEOUT);
}

main( ) {
actorSelf(&actorCap);
printf("Hello from the main thread [%d] ", threadSelf( ));
fprintCap(stdout, "in target actor", &actorCap);
printf("Address of annexThread function: %p\n", annexThread);
threadDelay(K_NOTIMEOUT);
}
8.5. Creating and deleting a thread 221
-2-> neon targetActor_u
started aid = 23
Hello from the main thread [9] in target actor: 200000a6 869da80a 17 1
Address of annexThread function: 0xffff6020
We use the following command which calls the newThreadAttr function to
create a thread in the previous actor. This thread will execute the annexThread
function de ned in the target actor. As shown by the results, this function has
virtual address 0xffff6020:
-1-> cat $CHORUS/sources/threads/creatingActor/creatingActor.c
#include "utilities.h"
KnCap targetActorCap;
KnPc entryPoint;
KnDefaultStartInfo_f startInfo;
KnThreadLid newThreadLid;
main(int argc, char *argv[ ]) {
/* getting the actor capability and the entry point */
readCap(argv + 1, &targetActorCap);
sscanf(argv[5], "%x", &entryPoint);
newThreadLid = newThreadAttr(&targetActorCap, (KnPc) entryPoint,
K_ACTIVE, 0x2000, K_SCHED_FIFO, 150);
if(newThreadLid < 0) {
fprintf(stderr, "error on newThreadAttr: %s\n",
strSysError(newThreadLid));
exit(2);
}
printf("New thread %d created", newThreadLid);
fprintCap(stdout, " in actor", &targetActorCap);
}
-1-> neon creatingActor_u 200000a6 869da80a 17 1 0xffff6020
started aid = 2
New thread 7 created in actor: 200000a6 869da80a 17 1
-1->
The new thread executes and prints its results:
+++ Entering annexThread function in target actor: 200000a6 869da80a 17 1
+++ Executing thread 7 in actor : 200000a6 869da80a 17 1

We can also check that the target actor contains two threads before killing it:
-1-> rsh neon arun cs -la 23
started aid = 2
ChorusOS r4.0.0 Site 0 Time 1d 14h 28m 1
....... LID .... TH# NAME
....... 0023 .... 002 targetActor_u
222 Chapter 8. Threads
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0007 150 00000000 00000000 ff6380 0- 1- 0
0009 140 00000000 00000000 ff6018 0- 1- 0 main
...........................
-1-> rsh neon akill 23
-1->
d) Creating a thread executing a function with parameters
In the previous examples, we created threads executing functions which were
not expecting parameters; the threadCreate call does not o er this service by
itself but lets the programmer do so if required.
Before giving the code of a newThreadArg function, which creates a new thread
executing a function with a variable number of arguments, we brie y describe
the organization of the user stack of a thread on the systems that we use (ix86
platforms). This stack is used to keep track of function calls during the execution
of the thread. When a function is invoked, an activation record or frame is
pushed onto that stack and the frame related to a function call is popped o
when returning from the call. This frame contains the following information for
void functions calls:
 the value of actual parameters as de ned by the calling function. Parameters
are pushed rst onto the stack in reverse order;
 the return address where execution will resume when returning from the call;
 a link to the frame corresponding to the caller;
 local variables (corresponding to automatic variables in C).
We can now give the source of a newThreadArg function that we add to the
utilities library. The prototype of this function
KnThreadLid newThreadArg(
KnPc function, /* entry point (a function) */
int status, /* thread's status: K_ACTIVE or K_INACTIVE */
char *symbName, /* thread's symbolic name */
int nbArg, /* number or parameters */
...
);
is added in the utilities.h header le.
This function creates a new thread executing a function with an arbitrary num-
ber of arguments, it is limited to user threads. In this code we use the set
of macros de ned in the stdarg.h le which allow us to write functions with
a variable number of parameters. The nbArg parameter of the newThreadArg
function de nes to the number of parameters of the function that the new thread
will execute, these parameters correspond to the ... ellipse notation. We use
long as a generic type of these arguments:
8.5. Creating and deleting a thread 223
--> cat $CHORUS/sources/utilities/lib/newThreadArg.c
#include "utilities.h"
#include <stdarg.h>
#define USER_STACK_SIZE 0x4000

KnThreadLid newThreadArg(KnPc entryPoint, int status,


char *symbName, int nbArg, ...) {
KnActorPrivilege actorPriv;
KnRgnDesc rgnDesc;
KnDefaultStartInfo_f startInfo;
long *userStack, arg;
KnThreadLid childLident;
va_list p_list;
int result, ind;
/* check if the actor is a user actor */
result = actorPrivilege(K_MYACTOR, &actorPriv, NULL);
if(result != K_OK) {
fprintf(stderr, "Error on actorPrivilege : %d\n", result);
return -1; }
if (actorPriv == K_SUPACTOR) {
fprintf(stderr, "Error: only for user actors\n");
return -1; }
/* set default startInfo fields */
startInfo.dsType = K_DEFAULT_START_INFO;
startInfo.dsSystemStackSize = K_DEFAULT_STACK_SIZE;
startInfo.dsPrivilege = K_USERTHREAD;
/* allocation of the user stack as a new region */
rgnDesc.startAddr = 0x0;
rgnDesc.size = USER_STACK_SIZE;
rgnDesc.options = K_ANYWHERE | K_FILLZERO | K_WRITABLE;
result = rgnAllocate(K_MYACTOR, &rgnDesc);
if (result != K_OK) {
fprintf(stderr, "error on rgnAllocate: %s\n", strSysError(result));
return -1;
}
userStack = (long *) (rgnDesc.startAddr + USER_STACK_SIZE
- nbArg * sizeof(long));
va_start(p_list, nbArg);
for(ind = 0; ind < nbArg; ind ++) {
*userStack = va_arg(p_list, long);
userStack ++;
}
startInfo.dsUserStackPointer = (long *) (rgnDesc.startAddr +
USER_STACK_SIZE - (nbArg + 1) * sizeof(long));
va_start(p_list, nbArg);
/* set entry point for the new thread */
startInfo.dsEntry = entryPoint;
224 Chapter 8. Threads
/* create new thread in state status */
result = threadCreate(K_MYACTOR, &childLident, status, NULL,
&startInfo);
if (result == K_OK) {
if(symbName != NULL)
threadName(K_MYACTOR, childLident, NULL, symbName);
return childLident;
}
else {
fprintf(stderr, "error on threadCreate: %d\n", strSysError(result));
rgnDesc.options = 0;
rgnFree(K_MYACTOR, &rgnDesc);
return -1;
}
}

The next command uses this function to create a new thread executing a func-
tion with 3 long parameters. The function prints the addresses and values of
its parameters and of its local variables before entering an in nite delay:
--> cat $CHORUS/sources/threads/paramTh/paramTh.c
#include "utilities.h"
void annexThread(int a, int b, int c) {
int t[2] = {16, 256};
int x = 1024;
fprintf(stderr, "address of x: %p\n", &x);
fprintf(stderr, "addresses of t[0] and t[1]: %p, %p\n", t, t + 1);
fprintf(stderr, "a : address = %p value = %d\n", &a, a);
fprintf(stderr, "b : address = %p value = %d\n", &b, b);
fprintf(stderr, "c : address = %p value = %d\n", &c, c);
threadDelay(K_NOTIMEOUT);
}
main( ) {
KnThreadLid annexThreadLid;
annexThreadLid = newThreadArg((KnPc) annexThread, K_ACTIVE,
"annex", 3, 250, 35, -13);
if(annexThreadLid == -1) {
fprintf(stderr, "Error on newThreadParam\n");
exit(2);
}
fprintf(stderr, "Thread %d has been created\n", annexThreadLid);
threadDelay(K_NOTIMEOUT);
--> neon-n paramTh_u &
[1] 15548
--> started aid = 23
Thread 7 has been created
address of x: 0xfffedfe0
addresses of t[0] and t[1]: 0xfffedfe4, 0xfffedfe8
8.5. Creating and deleting a thread 225
a : address = 0xfffedff4 value = 250
b : address = 0xfffedff8 value = 35
c : address = 0xfffedffc value = -13
--> rsh neon arun cs -la 23; rsh neon akill 23
started aid = 2
ChorusOS r4.0.0 Site 0 Time 1d 16h 39m 32
ACTOR-UI KEY LID TYPE STATUS TH# NAME
200000ba 869da80a 00000017 0000067b 0023 USER STARTED 002 paramTh_u
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0007 140 00000000 00000000 ff6380 0- 1- 0 annex
0009 140 00000000 00000000 ff6018 0- 1- 0 main
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
200000ba 869da80a def no fac0ac 0 200000ba 869da80a
START SIZE OFFSET ALLOC OPTIONS
fffea000 0000c000 00000000 0000c000 WR
ffff6000 00009000 00000000 00009000 EX
-->

8.5.2 Deleting a thread


8.5.2.1 The e ect of calling exit and exit
We have seen in chapter 6 that by calling the exit or exit functions, we could
terminate a c actor. This type of call deletes the actor and all its resources
are freed. As a consequence, all the threads of the actor are deleted. In the
following example, we can see that the annex thread ( fo scheduling class and
same priority as the main thread) never executes. Once it has been created, the
main thread exits and thus deletes the actor:
--> cat $CHORUS/sources/threads/threadExit/threadExit.c
#include "utilities.h"
annexThread( ) {
printf(" I am the new thread %d\n", threadSelf( ));
/* entering an infinite delay */
threadDelay(K_NOTIMEOUT);
}
main( ) {
KnThreadLid threadLi;
threadLi = newThread((KnPc) annexThread, K_ACTIVE, "annex");
printf("A new thread has been created. Its local identifier is %d\n",
threadLi);
exit(0); /* return would have the same effect */
}
--> neon threadExit_u
started aid = 23
A new thread has been created. Its local identifier is 7
-->
226 Chapter 8. Threads
Any thread of an actor may call exit or exit which implies the termination
of the whole actor. In the following example we created a thread with a higher
priority which exits. We can observe that the main thread is not scheduled after
the annex thread exits: it no longer exists as its actor has been deleted.
--> cat $CHORUS/sources/threads/threadExitModified/threadExitModified.c
#include "utilities.h"
annexThread( ) {
printf(" I am the new thread %d\n", threadSelf( ));
exit(0);
}
main( ) {
KnThreadLid threadLi;
threadLi = newThreadAttr(K_MYACTOR, (KnPc) annexThread, K_ACTIVE,
4096, K_SCHED_FIFO, 130);
printf("A new thread has been created. Its local identifier is %d\n",
threadLi);
}
--> neon threadExitModified_u
started aid = 23
I am the new thread 9
-->

8.5.2.2 The threadDelete primitive


The exit and exit functions act on all the threads of a c actor by deleting this
actor. In many situations it may be necessary to terminate a speci c thread
without terminating the other threads of the actor.
This operation is performed by calling
int threadDelete(
KnCap *actorCap,
KnThreadLid threadLi
);

where actorCap identi es the actor the thread belongs to and where the thread
is identi ed by threadLi. If the value of actorCap is K MYACTOR and the value of
threadLi is K MYSELF, then the current thread is deleted.
8.5.2.3 Examples
a) One of the most important use of the threadDelete primitive concerns the
termination of annex threads created in an actor.
Let us consider the following example where an annex thread is created and
simply returns after having printed a message:
8.5. Creating and deleting a thread 227
--> cat $CHORUS/sources/threads/threadRet/threadRet.c
#include "utilities.h"
annexThread( ) {
printf(" I am the new thread %d\n", threadSelf( ));
}

main( ) {
KnThreadLid threadLi;
threadLi = newThread((KnPc) annexThread, K_ACTIVE, "annex");
printf("A new thread has been created. Its local identifier is %d\n",
threadLi);
threadDelay(K_NOTIMEOUT);
}
Execution of this code leads to the following:
--> neon threadRet_u
started aid = 23
A new thread has been created. Its local identifier is 9
Segmentation fault thread 9 PC 0 faultAddr 0
I am the new thread 9
-->
The stack's frame relating to the initial function's call performed when activat-
ing a thread has a null return point eld. Thus, when trying to return from that
level, execution leads to an exception (segmentation fault) which terminates the
actor. The function called when starting the annex thread should terminate the
thread instead of returning. It is therefore imperative to call
threadDelete(K_MYACTOR, K_MYSELF);

instead of simply returning from the function. This modi cation leads to a
correct execution of the corresponding threadRet2 application:
--> neon-n threadRet2_u &
[1] 15828
--> started aid = 23
A new thread has been created. Its local identifier is 9
I am the new thread 9
The results of the cs utility show that the actor still exists and only contains
its main thread:
--> rsh neon arun cs -la 23
started aid = 2
ChorusOS r4.0.0 Site 0 Time 1d 18h 19m 14
ACTOR-UI KEY LID TYPE STATUS TH# NAME
200000c3 869da80a 00000017 00000001 0023 USER STARTED 001 threadRet2_u
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0007 140 00000000 00000000 ff6380 0- 1- 0 main
.........................
228 Chapter 8. Threads
b) The next example shows that the main thread of an actor may be
deleted by a call to threadDelete without a ecting the other threads
belonging to that actor:
--> cat $CHORUS/sources/threads/threadDel/threadDel.c
#include "utilities.h"
annexThread( ) {
printf(" I am the new thread %d\n", threadSelf( ));
threadDelay(K_NOTIMEOUT);
}
main( ) {
KnThreadLid threadLi;
threadLi = newThread((KnPc) annexThread, K_ACTIVE, "annex");
printf("A new thread has been created. Its local identifier is %d\n",
threadLi);
threadDelete(K_MYACTOR, K_MYSELF);
}
--> neon-n threadDel_u &
[1] 15869
--> started aid = 2
A new thread has been created. Its local identifier is 8
I am the new thread 8
We can observe that the c actor still exists and just contains the annex thread:
--> rsh neon arun cs -la 2
started aid = 22
ChorusOS r4.0.0 Site 0 Time 1d 18h 26m 8
ACTOR-UI KEY LID TYPE STATUS TH# NAME
200000c5 869da80a 00000002 000006c3 0002 USER STARTED 001 threadDel_u
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0008 140 00000000 00000000 ff61cc 0- 1- 0 annex
..............................

c) Another important use of the threadDelete primitive is related to the fact


that an actor still exists even if all its threads have been deleted:
--> cat $CHORUS/sources/threads/threadDelAll/threadDelAll.c
#include "utilities.h"
annexThread( ) {
printf(" I am the new thread %d\n", threadSelf( ));
threadDelete(K_MYACTOR, K_MYSELF);
}
main( ) {
KnThreadLid threadLi;
threadLi = newThread((KnPc) annexThread, K_ACTIVE, "annex");
printf("A new thread has been created. Its local identifier is %d\n",
threadLi);
threadDelete(K_MYACTOR, K_MYSELF);
}
8.6. Activating a thread 229
--> neon-n threadDelAll_u &
[1] 15922
--> started aid = 22
A new thread has been created. Its local identifier is 9
I am the new thread 9
--> rsh neon arun cs -la 22
started aid = 24
ChorusOS r4.0.0 Site 0 Time 1d 18h 32m 20
ACTOR-UI KEY LID TYPE STATUS TH# NAME
200000c8 869da80a 00000016 000006d3 0022 USER STARTED 000 threadDelAll_u
..................................
The ability to keep an actor without any threads alive is a commonly used fea-
ture:
 if we look at the results of the cs command, we can observe that various
supervisor actors created at boot time (AM or ADMIN for instance) have no
threads. They contain the de nitions of data and functions which are invoked
by threads of other user or supervisor actors through local access points (laps -
see chapter 11);
 the case of the SHM REP actor we already mentioned in 7.8.1 is even more
symptomatic. When created, it is an empty user actor acting as a repository
for shared memory segments (POSIX-SHM). During its life, no thread is is ever
created in it, nor does a thread execute in its context; in fact it does not contain
any code to be executed.
Such an actor has to be deleted by calling the akill command or through one
of the akill or actorDelete primitives.
8.6 Activating a thread
We have seen that a thread can be created in an inactive state by using the
K INACTIVE value for the status parameter of the threadCreate call. This
thread may then be activated later on by calling the primitive
#include <exec/chExec.h>
int threadActivate(
KnCap *actorCap,
KnThreadLid threadLi
);
After the call, the thread will become eligible for scheduling. If the thread is
not currently in the inactive state, the call has no e ect. When it succeeds the
call returns K OK, or a negative value if an error is encountered (K EINVAL for
an inconsistent capability or an invalid thread identi er, K EUNKNOWN for a non
reachable actor or K EFAULT for data outside the address space).
230 Chapter 8. Threads
8.7 Stopping and restarting a thread
The primitives we describe here should be used by system applications and
should not be used for synchronizing threads inside an application.

8.7.1 Stopping a thread


A thread may be stopped by a call to the primitive
int threadStop(
KnCap *actorCap,
KnThreadLid threadLi
);

It is worth noting that the e ect of the call may not be instantaneous if the
target thread is currently executing a system call. What is guaranteed in such
a case is that the thread will be stopped before returning from the call. The
call returns either K OK when it succeeds, or a negative error code (K EINVAL,
K EUNKNOWN or K EFAULT).

8.7.2 Restarting a stopped thread


A stopped thread will not be scheduled until it has been restarted by a call to
int threadStart(
KnCap *actorCap,
KnThreadLid threadLi
);

The function returns either K OK when it succeeds, or a negative error code


(K EINVAL, K EUNKNOWN or K EFAULT).

8.8 Aborting a thread


8.8.1 Introduction
A thread may be blocked when requesting kernel services if the corresponding
request cannot be satis ed immediately, for example when trying to read a
message on a port or trying to acquire a semaphore. Blocking and waking up
threads are basic operations performed by the kernel. It is possible to force a
thread to be awoken when it is blocked in a call by using a mechanism called
8.8. Aborting a thread 231
abort. When a blocked thread can be awoken it is said to be abortable. Some
calls are abortable, others are not. For example, a thread that is blocked by a
call to threadDelay(K NOTIMEOUT) is abortable, but if it is blocked by a call to
threadDelay(K NOTIMEOUT NOABORT) it is not. When a blocked thread on an
abortable call is awoken by an abort operation, the called function returns the
K EABORT value.

8.8.2 The complete states diagram of a thread


The following diagram describes the whole set of states of a thread:
INACTIVE STOPPED WAITING
K INACTIVE threadStop threadStart
wakeup
block threadAbort

threadCreate threadActivate

K ACTIVE
ACTIVE ACTIVE
ACTIVE EXTERNAL EXTERNAL
INTERNAL ABORT ABORTED
HANDLER

 the thread executes in either INTERNAL mode or EXTERNAL mode. In


the rst case execution is inside its home actor, in the second case, the execution
actor is di erent from the home actor. This type of situation is the result of a
cross actor invocation (LAP or abort handler);
 the thread is either ABORTED or NON ABORTED: a thread is in an
ABORTED state if a call to threadAbort has been done for it but the abortion
has not yet been consumed. It is handled immediately if the thread is currently
blocked on an ABORTABLE system, the thread is awoken and the call returns
the K EABORT value.
When a thread is in an ABORTED state, the abortion will be consumed in one
of the following situations:
- the thread issues an abortable system call, it will not be blocked on that
call and will return with the K EABORT value;
- the thread invokes the primitive threadAborted;
- an abort handler (see 14.2) has been attached to the thread's home actor
and the thread is executing in its home actor (or it returns from external
to internal mode).
232 Chapter 8. Threads
8.8.3 The threadAbort and threadAborted primitives
Abortion of a thread is achieved by a call to the function
int threadAbort(
KnCap *actorCap,
KnThreadLid threadLi
);
The e ect of this call depends on the thread's state:
 if the thread is currently blocked on an abortable call, the thread exits the
blocking state and the blocking system call returns the K EABORT error value;
 in any other case, the thread enters the aborted state. The abortion will
be consumed when the thread issues a blocking and abortable call (the thread
won't be blocked and the K EABORT value will be returned by the call).
The function returns K OK upon success, or a negative value if an error is en-
countered (K EINVAL, K EUNKNOWN or K EFAULT).
A call to the primitive
int threadAborted(void);
returns 1 if the thread is in an aborted state: in this case the abortion is
consumed (the thread exits the aborted state). If the thread is not in the
aborted state, the primitive returns 0.
8.8.4 Examples
8.8.4.1 A command for aborting a given thread
The following example requests the abortion of a thread whose identi cation is
given as an argument (the rst four parameters are the capability of the thread's
home actor and the fth argument is the local identi cation of the thread in
that actor):
--> cat $CHORUS/sources/threads/threadAbort/threadAbort.c
#include "utilities.h"
KnCap actorCap;
int result;
main(int argc, char *argv[ ]) {
if(argc != 6) {
fprintf(stderr, "bad call\n");
exit(1); }
readCap(argv + 1, &actorCap);
result = threadAbort(&actorCap, atoi(argv[5]));
if (result < 0)
fprintf(stderr, "error on threadAbort: %d\n", strSysError(result));
}

This application will be used a little further (see 8.8.4.3).


8.8. Aborting a thread 233
8.8.4.2 Thread's auto-abortion
In the following example, the thread requests abortion of itself before entering
a long loop and an in nite abortable delay:
--> cat $CHORUS/sources/threads/abortableDelay/abortableDelay.c
#include <chorus.h>
main( ) {
int ind, result;
threadAbort(K_MYACTOR, K_MYSELF);
printf("Before looping\n");
for(ind = 0; ind < 50000000; ind ++)
; /* long empty loop */
printf("After looping\n");
result = threadDelay(K_NOTIMEOUT);
if(result == K_EABORT)
printf("threadDelay aborted\n")
else
printf("result = %d\n", result);
}
--> neon abortableDelay_u
started aid = 23
Before looping
After looping
threadDelay aborted
-->
We can observe that the abortion request immediately awakes the thread when
it calls the threadDelay primitive. It has no e ect on the loop it executes
before then.
8.8.4.3 A non abortable call
We modi ed the previous code
- by deleting the request for auto-aborting the thread;
- by replacing the abortable call threadDelay(K NOTIMEOUT) by its equiv-
alent non-abortable one threadDelay(K NOTIMEOUT NOABORT).
to produce a notAbortDel application that we execute:
--> neon-n notAbortDel_u &
[1] 16123
--> started aid = 23
Before looping
After looping
We then request abortion of the thread by using the threadAbort application
we built. We can check that the request has no e ect and the thread or actor
has to be killed:
234 Chapter 8. Threads
--> rsh neon arun cs -la 23
started aid = 22
ChorusOS r4.0.0 Site 0 Time 1d 19h 14m 22
ACTOR-UI KEY LID TYPE STATUS TH# NAME
200000d0 869da80a 00000017 00000000 0023 USER STARTED 001 notAbortDel_u
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0007 140 00000430 00000430 ff6380 0- 1- 0 main
............................
--> neon threadAbort_u 200000d0 869da80a 17 0 7
started aid = 22
--> rsh neon aps grep notAbort
0 23 notAbortDel_u 0 N/A
--> rsh neon akill 23
-->

8.8.4.4 Testing the aborted state


Finally, we install an application abortedState which has the following source:
--> cat $CHORUS/sources/threads/abortedState/abortedState.c
#include <chorus.h>
main( ) {
int ind, result;
threadAbort(K_MYACTOR, K_MYSELF);
printf("Before looping\n");
for(ind = 0; ind < 100000; ind ++) ; /* looping */
printf("After looping\n");
threadAbort(K_MYACTOR, K_MYSELF);
result = threadAborted( );
if(result == 1)
printf("Aborted state\n");
else
printf("Non aborted\n");
result = threadAborted( );
if(result == 1)
printf("Aborted state\n");
else
printf("Non aborted\n");
}
--> neon abortedState_u
started aid = 23
Before looping
After looping
Aborted state
Non aborted
-->
We can observe that abortion requests are not cumulated, an abortion request
for a thread already in an aborted state is ignored.
8.9. Getting the attributes of all threads in an actor 235
8.9 Getting the attributes of all threads in an
actor
8.9.1 The threadStat primitive
An actor may contain an arbitrary number of threads. Thus, it may be interest-
ing to get the description of all threads it contains. The core executive provides
a primitive for that purpose.
Firstly, the structure KnThreadStat contains the following elds which describe
the attributes of a thread:
 tsLid with type KnThreadLid: the local identi er of the thread;
 tsStatus with type KnThreadStatus: the current status (K ACTIVE or K INAC-
TIVE) of the thread;
 tsSvStack with type VmAddr: the address of the thread's system stack (lo-
cated in supervisor address space);
 tsSvStackSize with type VmSize: the number of bytes allocated to the
thread's system stack.
A call to the primitive
#include <exec/chexec.h>
int threadStat(
KnCap *actorCap,
unsigned int options, /* always 0 */
KnThreadStat *stat,
unsigned int bu erSize
);
extracts information relative to the threads created in the named actor. The
stat argument points to a bu er of bu erSize bytes which will contain the de-
scriptions of the threads upon return from the call (no more than bu erSize
bytes will be written).
When the call is successful, the total number of threads in the actor is returned
by the function. A negative value is returned when an error occurs (K EINVAL,
K EUNKNOWN or K EFAULT).

8.9.2 Example
In this example, the main thread of the c actor dynamically creates 4 new
threads by calling the newThread function that we developed. The second
thread which is created (value 1 of ind) is created in the inactive state. All
other threads are active and execute the annex function. Once these di erent
threads have been created, the main thread calls the threadStat primitive
236 Chapter 8. Threads
and displays the number of threads in the actor, and for each thread, its local
identi er and its state:
--> cat $CHORUS/sources/threads/threadStat/threadStat.c
#include "utilities.h"
KnThreadStat stat[10];
void annex( ) {
printf(" *** I am the thread %d\n", threadSelf( ));
threadDelay(K_NOTIMEOUT);
}

main( ) {
int ind, result, state;
printf("I am the main thread %d\n", threadSelf( ));
/* create 4 new threads */
for(ind = 0; ind < 4; ind ++) {
/* if ind is 1, the thread is inactive */
state = (ind == 1) ? K_INACTIVE : K_ACTIVE;
result = newThread((KnPc) annex, state, NULL);
printf("New thread %d has been created\n", result);}
/* yield the processor: all active threads will execute */
threadDelay(K_NOBLOCK);
result = threadStat(K_MYACTOR, 0, stat, sizeof(stat));
printf("There are %d threads in the actor\n", result);
/* for each thread in the actor, display its local id and its state */
for(ind = 0; ind < result; ind ++) {
printf("Thread %d", stat[ind].tsLid);
printf(" ==> state is %d: %s\n", stat[ind].tsStatus,
(stat[ind].tsStatus == K_ACTIVE) ? "K_ACTIVE" : "K_INACTIVE");
}
threadDelay(K_NOTIMEOUT);
}
--> neon threadStat_u
started aid = 23
I am the main thread 7
New thread 8 has been created
New thread 9 has been created
New thread 10 has been created
New thread 11 has been created
*** I am the thread 8
*** I am the thread 10
*** I am the thread 11
There are 5 threads in the actor
Thread 11 ==> state is 0: K_ACTIVE
Thread 10 ==> state is 0: K_ACTIVE
Thread 9 ==> state is 2: K_INACTIVE
Thread 8 ==> state is 0: K_ACTIVE
Thread 7 ==> state is 0: K_ACTIVE
8.10. Execution times 237
Execution of the code shows that the inactive thread (whose local identi er
is 9) which was created in the K INACTIVE state is not scheduled (only three
messages *** I am the thread ... are displayed).
8.10 Execution times
When the VTIMER feature is active, it is possible to get, for each thread, the
times it consumed in internal mode (executing inside the actor where it was
created) and in external mode (executing in a cross-actor invocation).
The information can be accessed by calling the function
#include <vtimer/chVtimer.h>
int threadTimes(
KnCap *actorCap,
KnThreadLid threadLi,
KnTimeVal *internal,
KnTimeVal *external
);
If the parameter threadLi has the value K ALLACTORTHREADS, the function re-
turns the sum of execution times in each mode of all the threads belonging to
the actor identi ed by actorCap.
Upon success the function returns K OK. A negative error code is returned when
an error occurs (K EINVAL, K EUNKNOWN or K EFAULT).
In the following example, the main thread creates a new thread before entering
a busy loop. It then extracts information on execution time before yielding the
processor. At that time, the second thread executes: it enters a busy loop and
extracts information on execution times. When it terminates, the main thread
is scheduled and displays execution time information before terminating:
--> cat $CHORUS/sources/threads/threadTimes/threadTimes.c
#include "utilities.h"
#include <vtimer/chVtimer.h>

void annex( ) {
int ind;
KnTimeVal internal, external;
KnThreadLid localId;
localId = threadSelf( );
for(ind = 0; ind < 500000000; ind ++) ; /* a busy long loop */
threadTimes(K_MYACTOR, K_MYSELF, &internal, &external);
printf("th %d ==> internal=%ds.%d ns, external=%ds.%d ns\n",
localId, internal.tmSec, internal.tmNSec,
external.tmSec, external.tmNSec);
238 Chapter 8. Threads
threadTimes(K_MYACTOR, K_ALLACTORTHREADS, &internal, &external);
printf(
"th %d all threads ==> internal=%ds.%d ns, external=%ds.%d ns\n",
localId, internal.tmSec, internal.tmNSec,
external.tmSec, external.tmNSec);
threadDelete(K_MYACTOR, K_MYSELF);
}

main( ) {
int ind;
KnTimeVal internal, external;
KnThreadLid localId, annexId;
localId = threadSelf( );
printf("main thread's local id: %d\n", localId);
/* create a new active thread with no name */
annexId = newThread((KnPc)annex, K_ACTIVE, NULL);
for(ind = 0; ind < 500000000; ind ++) ; /* a busy loop */
/* get and print execution times */
threadTimes(K_MYACTOR, K_MYSELF, &internal, &external);
printf("th %d ==> internal=%ds.%d ns, external=%ds.%d ns\n",
localId, internal.tmSec, internal.tmNSec,
external.tmSec, external.tmNSec);
threadTimes(K_MYACTOR, K_ALLACTORTHREADS, &internal, &external);
printf(
"th %d all threads ==> internal=%ds.%d ns, external=%ds.%d ns\n",
localId, internal.tmSec, internal.tmNSec,
external.tmSec, external.tmNSec);
threadDelay(K_NOBLOCK); /* yield the processor */
threadTimes(K_MYACTOR, K_ALLACTORTHREADS, &internal, &external);
printf(
"th %d all threads ==> internal=%ds.%d ns, external=%ds.%d ns\n",
localId, internal.tmSec, internal.tmNSec, external.tmSec,
external.tmNSec);
}
--> rsh neon arun cs lM
............ VTIMER ....... (= the feature is active
--> neon threadTimes_u
started aid = 23
main thread's local id: 7
th 7 ==> internal=4s.300000000 ns, external=0s.0 ns
th 7 all threads ==> internal=4s.300000000 ns, external=0s.0 ns
th 8 ==> internal=4s.300000000 ns, external=0s.0 ns
th 8 all threads ==> internal=8s.600000000 ns, external=0s.0 ns
th 7 all threads ==> internal=8s.600000000 ns, external=0s.0 ns
-->
We will see in 12.4 that the VTIMER feature exports various other functions for
timing threads.
8.11. Managing a thread's context 239
8.11 Managing a thread's context
All threads have an execution context which has two components:
 a hardware context: it is machine dependent and is de ned as the set
of general purpose register values (stack pointer, program counter, . . . ). It is
pushed onto the stack when a trap, an exception or a preemption occurs;
 a software context de ned as two software registers (user privilege register
and supervisor privilege register). This context is used by the system itself to
provide the per-thread data service. Thus it is highly recommended not to use
it in applications.
The context of a given thread may be got or set by calling the primitive
#include <exec/chExec.h>
int threadContext(
KnCap *actorCap,
KnThreadLid threadLi,
unsigned int type,
void *oldContext,
void *newContext
);

The type argument may have the following values:


 K CURRCTX: oldContext and newContext must be pointers to KnThreadCtx
objects. As we said, the elds of the corresponding structure are machine
dependent;
 K SOFTCTX: oldContext and newContext must be pointers to objects of the
KnThreadSoftCtx type. The corresponding structure has two elds:

- long userReg : user privilege register;


- long supReg : supervisor privilege register;
 the thread's current context is copied at address oldContext, if it is not NULL,
in the address space of the calling thread;
 the new thread's context will be modi ed with the value pointed to by new-
Context, if it is not equal to NULL, in the address space of the calling thread.
It is important to note that the results of this type of operation are meaningful
only if the target thread is in a stable state (entering a trap, suspended, blocked,
for instance).
240 Chapter 8. Threads
8.12 A complete example: a fork function
8.12.1 Objectives and general principles
We intend to develop a fork function which can be called by any thread of an
actor and whose e ect is similar to the fork Unix system call (creation of a new
actor with duplication of regions of the current one and, creation of a "main"
thread in that actor by duplication of the calling thread).
Through di erent solutions of this example, we illustrate the creation of actors
as c actors or by invoking the low-level service, the duplication of the address
space of actors, the creation, stopping and activating of a thread, and scheduling
and manipulation of the hardware context of threads.
The general skeleton of the solution is illustrated below:
(2)rgnDup or
rgnMapFromActor or
rgnInitFromActor
address space address space
threadCreate (1) actorCreate
or acreate
(3) threadCreate
(4) threadContext

calling stub child


thread thread thread
calling actor new actor
The general principles and tools used to develop a solution are selected among
the following:
 when a thread calls the fork function, it rst creates a new thread in its actor
by calling the newThreadAttr function. This stub thread is intended to perform
the fork (creation of a new actor and a thread in that actor by duplication) by
calling the forkStub function. We use a stub thread to allow us to acquire a
stable context of the calling thread which will be duplicated in the new thread.
This new thread is in the active state: the calling thread has to release the
processor to this new thread, which should keep it until the fork operation has
been completed. Di erent solutions are possible, without using synchronization
mechanisms which will be presented in chapter 10:
- giving a higher priority to the stub thread;
- using the same priority for the stub thread and yielding the processor
(by calling threadDelay with the K NOBLOCK argument). This solution
works correctly if the SCHED FIFO scheduling policy is used. Otherwise
it may lead to an incorrect solution due to the load of the system and the
8.12. A complete example: a fork function 241
expiration of time quantum. To ensure correct execution, it is necessary
to stop the calling thread in the stub thread when it is activated;
- using the same priority for the stub thread and stopping the calling thread
with the threadStop primitive, this solution is the most secure as it does
not not depend on parameters like the scheduling policy, for example.
If the calling thread has stopped itself, it will be restarted by the stub thread
which calls threadStart before it terminates.
 to perform the fork, the stub thread will successively:
- either, if the ACTOR EXTENDED-MNGT feature is active, create an empty
c actor by calling acreate which allows a le context to be inherited
(standard input and output in particular) or create a new empty actor by
calling actorCreate;
- duplicate all the regions of the current actor; in particular, the user stack
of the calling thread will be copied during this duplication;
- create a new thread in the new actor: this thread is created in the inactive
state with the same priority as the calling thread and with a null stack
and a null entry point. Its context is then set by copying the hardware
context of the stopped calling thread: a side e ect of this operation is to
copy the stack, the stack's pointer and the instruction counter;
- restart the calling thread and activate the new one;
- terminate the stub thread by calling threadDelete;
 the fork function will return di erent values in the calling thread and in the
new one. This value may be the local identi er of the c actor newly created
by acreate (or the key head eld of the new actor's capability if it has been
created by actorCreate) in the calling thread and 0 in the new one.
8.12.2 Variations on solutions
In the various solutions:
 di erent messages are printed (when that is possible) to the standard error
output in order to trace the execution;
 di erent variables (static external or automatic) are initialized before the fork
operations to transmit information from the stub thread to the child thread.
8.12.2.1 A solution with creation of a c actor
In this solution, we made the following choices:
 after having created the stub thread with the same priority as its own priority
(newThread call), the calling thread stops itself (threadStop call);
242 Chapter 8. Threads
 the stub thread creates an empty c actor (acreate call): as we said in 6.4.4.1,
the c actor is created in the stopped state. The new c actor inherits the standard
input/output of the current actor which will allow us to display messages;
 all regions of the source actor are duplicated in the new c actor. This copy
is performed region by region (we will see that the API of the ON PAGE DEMAND
feature, which we will present in chapter 15, exports a function, rgnDup, which
duplicates the address space of an actor in another actor). Here, we will use
the copyRegions function whose code is given below (it uses the function
doRegionStat developed in 7.2.7.2):
#include "utilities.h"
int copyRegions(KnCap *sourceActor, KnCap *targetActor) {
int ind, result;
KnRgnStat statArray[20]; /* to get the descriptions of all regions */
static KnRgnDesc rgnDesc;
result = doRegionStat(sourceActor, statArray, 20);
if (result != K_OK)
exit(2);
for(ind = 0; ind < result; ind ++) {
rgnDesc.startAddr = statArray[ind].startAddr;
rgnDesc.size = statArray[ind].size;
rgnDesc.options = statArray[ind].options;
result = rgnInitFromActor(targetActor, &rgnDesc,
sourceActor, statArray + ind);
if (result != K_OK) {
fprintf(stderr, "error on rgnInitFromActor: %s\n",
strSysError(result));
return result; }
}
return(K_OK);
}
 the stub thread creates a child thread in the new c actor (threadCreate call)
and the attributes of this thread are the following:
- the thread is inactive (K INACTIVE ag);
- the scheduling attributes are inherited;
- no user stack is allocated (NULL argument) and the entry point is not
speci ed (NULL argument);
- the context of the new thread is set by copying the context of the calling
thread;
 the stub thread nally starts the new c actor by calling actorStart, activates
the new thread by a threadActivate call and restarts the calling thread by
calling threadStart;
8.12. A complete example: a fork function 243
 the return value of the fork call is 0 in the new thread and is the local
identi er of the new c actor in the calling thread.
--> cat $CHORUS/sources/threads/fork/fork.c
#include "utilities.h"
#include <am/afexec.h>
static int childActorLid, callingActorLid;
static KnThreadLid callingThreadLid;
static KnCap childActorCap, actorCap;

static void forkStub( ) {


KnThreadCtx context;
KnDefaultStartInfo_f startInfo;
KnThreadLid childThreadLid;
KnActorPrivilege actorPriv;
static AcParam param; /* all fields are initialized to zero */
int result = 0;
printf("Entering stubThread\n");
/* a new c_actor is created : it is initially empty */
actorPrivilege(K_MYACTOR, &actorPriv, NULL);
if (actorPriv == K_SUPACTOR) {
printf("calling actor is a supervisor actor\n");
return -1; }
param.acFlags = AFX_USER_SPACE;
param.acSite = 0;
childActorLid = acreate(&childActorCap, &param);
if(childActorLid == -1) {
perror("acreate");
threadStart(K_MYACTOR, callingThreadLid);
threadDelete(K_MYACTOR, K_MYSELF); }
printf("A new c_actor is created: %d\n", childActorLid);
fprintCap(stdout, "New c_actor's capability", &childActorCap);
/* a name is given to the new c_actor */
actorName(&childActorCap, NULL, "forkedActor");
/* all calling actor's regions are duplicated */
result = copyRegions(K_MYACTOR, &childActorCap);
if(result != K_OK) {
akill(&childActorCap);
childActorLid = -1;
threadStart(K_MYACTOR, callingThreadLid);
threadDelete(K_MYACTOR, K_MYSELF); }
/* a thread is created in the new actor with status K_INACTIVE */
/* a null stack, null entry point and inherited scheduling attrib */
startInfo.dsType = K_DEFAULT_START_INFO;
startInfo.dsSystemStackSize = K_DEFAULT_STACK_SIZE;
startInfo.dsUserStackPointer = NULL;
startInfo.dsEntry = NULL;
244 Chapter 8. Threads
startInfo.dsPrivilege = K_USERTHREAD;
result = threadCreate(&childActorCap, &childThreadLid, K_INACTIVE,
NULL, &startInfo);
if(result != K_OK) {
printf("error on threadCreate: %s\n",
strSysError(childThreadLid));
akill(&childActorCap);
childActorLid = -1;
threadStart(K_MYACTOR, callingThreadLid);
threadDelete(K_MYACTOR, K_MYSELF); }
printf("A thread %d is created in the new c_actor\n",
childThreadLid);
/* the calling thread's context is extracted */
result = threadContext(K_MYACTOR, callingThreadLid,
K_CURRCTX, &context, NULL);
/* the context of the new thread is set */
/* with the one of the stopped one */
result = threadContext(&childActorCap, childThreadLid,
K_CURRCTX, NULL, &context);
/* the new actor is started */
actorStart(&childActorCap);
/* the new thread of the new actor is activated */
threadActivate(&childActorCap, childThreadLid);
/* the calling thread is restarted */
result = threadStart(K_MYACTOR, callingThreadLid);
if(result != K_OK)
printf("error on threadStart: %s\n", strSysError(result));
/* the stub thread is deleted */
threadDelete(K_MYACTOR, K_MYSELF);
}

int fork( ) {
KnThreadLid stubThreadLid;
printf("entering fork by thread %d\n",
callingThreadLid = threadSelf( ));
/* a new thread is created for executing the fork */
stubThreadLid = newThread((KnPc)forkStub, K_ACTIVE, "stub");
printf("stub thread : %d\n", stubThreadLid);
callingActorLid = agetId( );
/* the calling thread stops */
threadStop(K_MYACTOR, K_MYSELF);
actorSelf(&actorCap);
if (uiEqual(&actorCap.ui, &childActorCap.ui) == 1) {
threadDelay(K_NOBLOCK);
return 0; /* it is the child c_actor */ }
else
return childActorCap.key.keyHead; /* it is the parent c_actor */
}
8.12. A complete example: a fork function 245
The callingFork application given below uses the fork service. Due to im-
plementation optimization leaning on private data, the threads must yield the
processor before terminating (by calling threadDelay):
--> cat $CHORUS/sources/threads/fork/callingFork.c
#include "utilities.h"
main( ) {
KnCap actorCap; int result;
actorSelf(&actorCap);
fprintCap(stdout, "Initial actor", &actorCap);
if ((result = fork( )) == 0) { /* it is the child */
actorSelf(&actorCap);
fprintf(stdout, "fork( ) is %d in thread %d", result, threadSelf( ));
fprintCap(stdout, " in actor", &actorCap);
threadDelay(K_NOBLOCK);
}
else { /* it is the parent */
actorSelf(&actorCap);
fprintf(stdout, "fork( ) is %d in thread %d", result, threadSelf( ));
fprintCap(stdout, " in actor", &actorCap);
threadDelay(K_NOBLOCK);
}
}
--> neon callingFork_u
started aid = 23
Initial actor: 20000121 869da80a 17 8dc
entering fork by thread 8
stub thread : 7
Entering stubThread
A new c_actor is created: 22
New c_actor's capability: 20000122 869da80a 16 0
A thread 9 is created in the new c_actor
capa: 20000122 869da80a 16 0
capa: 20000121 869da80a 17 8dc
fork( ) is 22 in thread 8 in actor: 20000121 869da80a 17 8dc
fork( ) is 0 in thread 9 in actor: 20000122 869da80a 16 0
-->

8.12.2.2 A solution with low level actor's creation


We rst simply changed, in the previous code, the sequence corresponding to
the creation of the child actor to the following
/* a new actor is created : it is initially empty */
actorPrivilege(K_MYACTOR, &actorPriv, NULL);
if (actorPriv == K_SUPACTOR) {
printf("calling actor is a supervisor actor\n");
return -1;
}
246 Chapter 8. Threads
result = actorCreate(K_MYACTOR, &childActorCap, K_USERACTOR, K_STOPPED);
if(result != K_OK) {
fprintf(stderr, "actorCreate: %s\n", strSysError(result));
threadStart(K_MYACTOR, callingThreadLid);
threadDelete(K_MYACTOR, K_MYSELF);
}
fprintCap(stdout, "New actor's capability", &childActorCap);
The callingFork2 application is identical to callingFork but it uses the new
version of the fork function
--> neon callingFork2_u
started aid = 24
Initial actor: 2000012a 869da80a 18 904
entering fork by thread 8
stub thread : 7
Entering stubThread
New actor's capability: 2000012b 869da80a 2 0
A new c_actor is created: 25
New c_actor's capability: 2000012c 869da80a 19 90c
A thread 9 is created in the new c_actor
fork( ) is 25 in thread 8 in actor: 2000012a 869da80a 18 904
fork( ) is 0 in thread 9 in actor: 2000012c 869da80a 19 90c
-->

Remark: some kind of miracle occurred: the actor was promoted to a c actor
and inherited the standard input/output les of the initial actor. This is due
to the use of private data which tends to mix up the two actors.
8.13 Posix threads (pthreads)
8.13.1 Introduction
The POSIX-THREADS library o ers a compatible implementation of the Posix
interface for thread management and synchronization.
A Posix thread-identi er (type pthread t) is equivalent to the Chorus local
identi er of the underlying Chorus thread.
We brie y present some of the functions of the corresponding API (a complete
presentation of pthreads programming can be found in the book of N. Bradfor
et al).
8.13.2 Posix threads' attributes
They are packed in a pthread attr t opaque structure. Examples of these
attributes and the constants corresponding to their default values are:
 the stack's size (PTHREAD STACK MIN);
8.13. Posix threads (pthreads) 247
 the stack's address (dynamically allocated);
 the detach state (PTHREAD CREATE JOINABLE);
 scheduling policy and priority.
This type of object can be globally initialized by a call to the function
#include <pthread.h>
int pthread attr init(pthread attr t *attr);
The e ect of the call is to give default values for every individual attribute.
Every attribute may be modi ed further by speci c calls:
 pthread attr setstacksize for modifying the stack's size eld;
 pthread attr setstackaddr for modifying the stack's address eld;
 pthread attr setdetachstate for modifying the detach status eld (le-
gal values for this status are PTHREAD CREATE JOINABLE and PTHREAD CRE-
ATE DETACHED).
A thread also has scheduling attributes:
 a scheduling policy: it is SCHED RR, SCHED FIFO or SCHED OTHER;
 a priority: it corresponds to the sole sched priority eld of a sched pa-
ram object.
These attributes may be set by calling the functions
 pthread attr setschedpolicy;
 pthread attr setschedparam.
8.13.3 Creating a pthread
A call to the function
#include <pthread.h>
int pthread create(
pthread t *thread,
const pthread attr t *attr,
void *(*start routine(void *)),
void *arg
);
creates a new thread with the attributes speci ed by the attr parameter in the
current actor. If the attr parameter has a NULL value, the default attributes
are used. If the call succeeds, the identi er of the new thread is stored at
address thread. The new thread will start execution by calling the function
start routine with arg as the only argument. Returning from that function will
be the equivalent for that thread of calling pthread exit with an argument
equal to the return value of the function.
248 Chapter 8. Threads
8.13.4 Yielding the processor
A thread may yield the processor by calling the function
#include <pthread.h>
void pthread yield(void);

The thread is queued at the end of the list of threads waiting to run relative to
its priority and the rst thread in that list is awoken.
8.13.5 Terminating a pthread
A call to the function
#include <pthread.h>
void pthread exit(void *status);

terminates the calling thread. An implicit call to that function is made when
a thread other than the main thread returns from the start routine. The value
passed as an argument to the function is used as the thread's exit status. It is
important to note that the thread (if di erent from the main thread) still exists
after calling pthread exit.
8.13.6 Killing a pthread
By a call to the function
#include <pthread.h>
int pthread kill(
pthread t thread,
int sig
);

the current thread may send a given signal to the thread with the identi cation
thread.
The value of sig may be:
 either SIGTHREADKILL: the target thread is deleted (the signal may not
be caught, ignored or blocked);
 or 0: no signal is sent. The call tests whether the target thread exists.
8.13. Posix threads (pthreads) 249
8.13.7 Example
--> cat $CHORUS/sources/threads/pthreads/pthread.c
#include <stdio.h>
#include <stdlib.h>
#include <chorus.h>
#include <pthread.h>
pthread_t pthread;
KnCap actorCap;

void pthread_function(char *arg) {


printf("A pthread has been created with chorus local identifier: %d\n",
threadSelf( ));
printf("Function called with argument: %s\n", arg);
printf("pthread is exiting\n");
pthread_exit(NULL);
}

main( ) {
int result;
actorSelf(&actorCap);
fprintCap(stdout, "actor", &actorCap);
printf("Hello I am %d and I'm trying to create a pthread\n", threadSelf( ));
result = pthread_create(&pthread, NULL, pthread_function,
"argument for the pthread");
if(result != 0) {
fprintf(stderr, "error on pthread_create: %d\n", result);
exit(1);
}
printf("Success !!! Pthread number %d has been created\n", pthread);
printf("Main thread yields the processor ...\n");
pthread_yield( ); /* yield the processor */
printf("Main thread enters threadDelay ...\n");
threadDelay(K_NOTIMEOUT);
printf("Main thread is terminating ...\n");
}
-2-> neon pthread_u
started aid = 25
actor: 20000136 869da80a 19 948
Hello I am 9 and I'm trying to create a pthread
Success !!! Pthread number 8 has been created
Main thread yields the processor ...
A pthread has been created with chorus local identifier: 8
Function called with argument: argument for the pthread
pthread is exiting
Main thread enters threadDelay ...
250 Chapter 8. Threads
-1-> rsh neon arun cs -la 25
started aid = 24
ChorusOS r4.0.0 Site 0 Time 2d 12h 37m 13
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000136 869da80a 00000019 00000948 0025 USER STARTED 002 pthread_u
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0008 140 00000000 00000000 ff61cc ff- 1- 0
0009 140 00000000 00000000 ff6018 0- 1- 0 main
.........................
-1-> neon threadAbort_u 20000136 869da80a 19 948 9
started aid = 24
-1->

Main thread is terminating ...


-2->
Chapter 9
Synchronization
9.1 Introduction
First of all, the core executive provides two basic synchronization services called
thread semaphores and mutexes (mutexes used to be provided by a con g-
urable feature within previous versions of the system). In addition the following
features may be con gured:
 RTMUTEX: management of real-time mutexes;
 SEM: management of general semaphores for synchronization;
 EVENT: management of event ag sets.
A common property of the di erent objects used for synchronization is that
they are allocated in the address space of actors. Thus,
 there is no limit imposed by the system on the number of these objects since
they are allocated in the address space of actors;
 threads belonging to di erent user actors may synchronize as soon as they
share regions where the synchronizing objects are located;
 it is not possible to synchronize a user actor and a supervisor actor;
 thread semaphores may be used for synchronizing threads of supervisor actors:
the sharing of the semaphore must be performed by using the address of the
thread semaphore in the supervisor address space directly, and not by mapping
the region's location in the di erent actors.
The executive module also includes a module (Driver/Kernel Interface) which
provides other mechanisms for creating short critical sections which are partic-
ularly adapted to the writing of drivers.
Finally, the POSIX-THREADS feature includes various synchronization mecha-
nisms such as joining threads, Posix mutexes, Posix semaphores and condition
variables.
251
252 Chapter 9. Synchronization
9.2 Thread semaphores
9.2.1 Overview
First of all, the core executive implements thread semaphores as a basic service.
Thread semaphores are a restricted form of binary semaphores usually named
private semaphores (only one thread may acquire it, other threads may post it).
As binary semaphores, thread semaphores have two possible states (POSTED
and UNPOSTED). They are allocated in actors' address spaces and correspond
to the KnThSem opaque type. The main advantage of this service is that it is
directly provided by the core executive (no additional module is necessary) and
implementation has been optimized.
We now summarize more precisely the requirements that applications need to
respect in order to guarantee a correct use of the mechanism:
 a thread semaphore should be initialized before use by just one thread, (by
calling the threadSemInit primitive) which then becomes the owning thread
of the thread semaphore;
 the owning thread of a thread semaphore should be the only thread which
tries to get the semaphore by calling the threadSemWait primitive, and should
therefore be the only thread which can be blocked on the semaphore when it is
in the UNPOSTED state.
Any thread knowing the thread semaphore can signal it, which means the own-
ing thread will be awoken if it is currently blocked on the semaphore, otherwise
the semaphore will be set to the POSTED state if the owning thread is not
currently blocked.
9.2.2 Initializing a thread semaphore
Before it can be used, a thread semaphore has to be initialized by a thread
which becomes the owner of the thread semaphore. A call to the primitive
#include <exec/chExec.h>
int threadSemInit(KnThSem *semaphore);

has the following e ects:


 the calling thread becomes the owner of the semaphore and it will be the only
thread allowed to perform calls to threadSemWait;
 the semaphore is set to the UNPOSTED state.
The function returns K OK when it succeeds and a negative value if it fails
(K EFAULT when semaphore points outside the address space).
9.2. Thread semaphores 253
9.2.3 Posting a thread semaphore
Any thread knowing a thread semaphore can set its value to POSTED by calling
the function
#include <exec/chExec.h>
int threadSemPost(KnThSem *semaphore);

If the thread owning the semaphore is currently blocked in a threadSemWait


call, it is awoken and the state of the semaphore is not changed (it is UN-
POSTED). Otherwise, the new state of the semaphore is POSTED (a call has
no e ect if the target semaphore's state is already POSTED).
The function returns K OK upon success and a negative error value otherwise:
K EFAULT if semaphore points outside the address space or K EINVAL if the
semaphore has not been correctly initialized.

9.2.4 Waiting on a thread semaphore


The thread owning a thread semaphore can call the primitive
#include <exec/chExec.h>
int threadSemWait(
KnThSem *semaphore,
KnTimeVal *waitLimit
);

If the state of the thread semaphore is POSTED, the thread is not blocked (the
K_OK value is returned by the function and the thread semaphore's new state
is UNPOSTED). If the semaphore is in the UNPOSTED state, the thread is
blocked. The waitLimit parameter de nes a timeout.
If a thread other than the owner performs this type of call, unspeci ed be-
havior results. This choice has been made to minimize the overhead of thread
semaphores.
If the function returns due to the POSTED value of the semaphore, K OK is
returned. Otherwise, a negative error code is returned:
value of errno error type
K EINVAL semaphore not correctly initialized or invalid waitLimit
K EFAULT semaphore outside the address space
K ETIMEOUT timeout occurred
K EABORT the call has been aborted
254 Chapter 9. Synchronization
9.2.5 Example
In the following example the main thread and a dynamically created thread of
the actor execute a compute function concurrently. The main thread can only
terminate and delete the actor when the other thread has nished its compu-
tation. Thus, the main thread uses a thread semaphore and when it nishes
its computation, calls threadSemWait. When it nishes its own computation,
the other thread in turn posts the semaphore to signal its termination; thus
the main thread may then exit. When the outcome command is called with
parameters, the main thread enters the round robin scheduling class.
In the code given below we use a function whose prototype is
int compute(int);
This will be presented in 12.3.4. It uses the services o ered by the VTIMER
feature, allowing the de nition of a virtual timeout which interrupts an in nite
loop and requires a dedicated supervisor server to execute the serverCompute
application. Here we assume that the compute function has been integrated
within the libutilities.a library. We shall simply state calling the function
simulates a computation whose duration is a random number of milliseconds in
the range [0..10000]. If the parameter is not 0, a message is printed when
entering and exiting the function.
--> cat $CHORUS/sources/synchronization/threadSem/threadSem.c
#include "utilities.h"
KnThSem thSem;
KnCap actorCap;
void annex( ) {
int result;
compute(1); /* <-- computation .... */
fprintf(stderr, "thread %d posts the semaphore\n", threadSelf( ));
result = threadSemPost(&thSem);
fprintf(stderr, "in thread %d threadSemPost returns %d\n",
threadSelf( ), result);
fprintf(stderr, "thread %d terminates\n", threadSelf( ));
threadDelete(K_MYACTOR, K_MYSELF);
}
main(int argc) {
int result;
KnThreadLid threadLi;
KnThreadDefaultSched schedAttr;
actorSelf(&actorCap);
if (argc != 1) {
threadScheduler(K_MYACTOR, K_MYSELF, &schedAttr, NULL);
schedAttr.tdClass = K_SCHED_RR;
threadScheduler(K_MYACTOR, K_MYSELF, NULL, &schedAttr);
}
9.2. Thread semaphores 255
/* initialize the thread semaphore */
result = threadSemInit(&thSem);
fprintf(stderr, "main thread (%d): threadSemInit = %d\n",
threadSelf( ), result);
/* create a new thread with the newThread function */
threadLi = newThread((KnPc)annex, K_ACTIVE, "annex");
if(threadLi < 0)
exit(1);
fprintf(stderr, "main thread: new thread %d is created\n", threadLi);
compute(1); /* <-- computation .... */
fprintf(stderr, "main thread: threadSemWait is called\n");
result = threadSemWait(&thSem, K_NOTIMEOUT);
fprintf(stderr, "main thread: threadSemWait returns %d\n", result);
}

We rst execute the application with the FIFO scheduling policy:


--> rsh neon arun cs -lM
started aid = 25
ChorusOS r4.0.0 Site 0 Time 2d 17h 23m 15
CORE SCHED_CLASS [ FIFO RR RT ] ............... VTIMER .......
--> neon-n serverCompute_s.r & (= activate the server
[1] 3580
--> started aid = 25

--> neon threadSem_u


started aid = 24
main thread (13): threadSemInit = 0
main thread: new thread 14 is created
From compute: thread 13 in actor 24 starts computation
From compute: thread 13 in actor 24 ends computation
main thread: threadSemWait is called (= main thread blocks
From compute: thread 14 in actor 24 starts computation
From compute: thread 14 in actor 24 ends computation
thread 14 posts the semaphore
in thread 14 threadSemPost returns 0
thread 14 terminates
main thread: threadSemWait returns 0
-->

The next set of results correspond to two di erent executions within the round
robin scheduling policy. We can observe, in the rst set, round robin in the
computation step and that the main thread waits for the semaphore:
--> neon threadSem_u RR
started aid = 24
main thread (13): threadSemInit = 0
main thread: new thread 14 is created
From compute: thread 13 in actor 24 starts computation
From compute: thread 14 in actor 24 starts computation
256 Chapter 9. Synchronization
From compute: thread 13 in actor 24 ends computation
main thread: threadSemWait is called (=
main thread blocks
From compute: thread 14 in actor 24 ends computation
thread 14 posts the semaphore
in thread 14 threadSemPost returns 0
thread 14 terminates
main thread: threadSemWait returns 0
-->
In the second execution, the new thread posts the semaphore before the main
thread blocks when calling threadSemWait. Thus, when it calls this function,
the semaphore is already posted, the thread is not blocked (it returns immedi-
ately from the call) and terminates:
--> neon threadSem_u RR
started aid = 24
main thread (13): threadSemInit = 0
main thread: new thread 14 is created
From compute: thread 13 in actor 24 starts computation
From compute: thread 14 in actor 24 starts computation
From compute: thread 14 in actor 24 ends computation
thread 14 posts the semaphore
in thread 14 threadSemPost returns 0
thread 14 terminates
From compute: thread 13 in actor 24 ends computation
main thread: threadSemWait is called
main thread: threadSemWait returns 0
-->

9.3 General mutexes


9.3.1 Overview
The core executive provides another synchronization mechanism, mutexes. In
previous releases these were a con gurable feature, but as mutexes are used to
synchronize system actors they have been integrated into the nucleus.
A mutex is a binary semaphore: it has only two possible values denoted by free
and locked. These objects are particularly adapted to the protection of critical
sections and the mutual exclusion of threads' execution. As with all objects
described in this chapter, mutexes are allocated in the user memory and are of
the type KnMutex.
9.3.2 Initializing a mutex
Once allocated, a mutex is initialized to the value free by calling the primitive
#include <sync/chMutex.h>
int mutexInit(KnMutex *mutex);
9.3. General mutexes 257
A successful call returns K OK. The K EFAULT error value is returned if mutex
points outside the address space.
When it is statically allocated, a mutex may be directly initialized to the free
value by calling the K KNMUTEX INITIALIZER macro as follows:
KnMutex mutex = K_KNMUTEX_INITIALIZER;

9.3.3 Acquiring a mutex


A mutex may be acquired by calling the function
#include <sync/chMutex.h>
int mutexGet(KnMutex *mutex);
If the mutex is currently locked, the calling thread is blocked until the mutex is
released. This type of call is non abortable and returns K OK except if mutex
points outside the address space, in which case K EFAULT is returned.
A non blocking form also exists:
#include <sync/chMutex.h>
int mutexTry(KnMutex *mutex);
This call has the same e ect as mutexGet except that if the mutex is currently
locked, the calling thread is not blocked: the call returns the value 0 if the
mutex was already locked and the value 1 if it was free.
9.3.4 Freeing a mutex
A mutex can freed it by by using the primitive
#include <sync/chMutex.h>
int mutexRel(KnMutex *mutex);
When the mutex is released, if there are blocked threads on a mutexGet call,
one of them is awoken.
Remark A deadlock will occur if a thread attempts to acquire a mutex that
it previously locked and a thread may free a mutex even if it did not acquire it
before.
9.3.5 Example
In this example, the main thread, running with a priority , rst creates two
new threads by using the newThreadAttr function:
 the rst one, named th1, calls the annex1 function and runs with priority
+ 10;
 the second one, named th2, calls the annex2 function and runs with priority
+ 20.
258 Chapter 9. Synchronization
By using a delay in the main and th1 thread, we ensure that th2 is scheduled,
although it has the lowest priority. Thus, it can get the mutex and enters its
computation step (corresponding to the call of the compute function we already
used in the previous example). When the one second delay in threads th1 and
main expires, the thread th2 is preempted since the th1 and main threads are
ready to run and have a higher priority. The main thread has the higher priority
but it is blocked as the mutex is already locked by th2. Thus, th1 is scheduled
and enters its loop. When it terminates, the th2 thread resumes its execution:
it terminates its loop and releases the mutex. The main thread is then awoken.
When it enters its last threadDelay call, th2 is scheduled and terminates. The
complete execution of the three threads is illustrated as follows:
mutexGet thread is awaken thread terminates
and is scheduled
delay (1+) s the thread is blocked
main
thread is preempted thread is scheduled thread is scheduled
mutexGet
by th1 mutexRel and terminates
thread is preempted
Running by main thread
th2
thread is scheduled thread terminates
delay 1 s
th1
--> cat $CHORUS/sources/synchronization/mutex/mutex.c
#include "utilities.h"
#include <sync/chMutex.h>
KnMutex mutex;

void annex1( ) {
int ind;
KnTimeVal delay;
fprintf(stderr, "Entering first thread [%d]\n", threadSelf( ));
/* waiting for 1 second */
K_MILLI_TO_TIMEVAL(&delay, 1000);
threadDelay(&delay);
compute(1); /* computation step */
threadDelete(K_MYACTOR, K_MYSELF);
}

void annex2( ) {
int ind;
fprintf(stderr, "Entering second thread [%d]\n", threadSelf( ));
mutexGet(&mutex);
compute(1); /* computation step */
mutexRel(&mutex);
fprintf(stderr, "Second thread [%d] after mutexRel\n", threadSelf( ));
threadDelete(K_MYACTOR, K_MYSELF);
}
9.4. Real-time mutexes 259
main( ) {
int ind;
KnThreadLid lid1, lid2;
KnTimeVal delay;
KnThreadDefaultSched schedParam;
/* initialize the mutex */
mutexInit(&mutex);
/* get scheduling attributes */
threadScheduler(K_MYACTOR, K_MYSELF, &schedParam, NULL);
/* create a new thread with priority = myPriority + 10 */
lid1 = newThreadAttr(K_MYACTOR, (KnPc) annex1, K_ACTIVE, 0x4000,
schedParam.tdClass, schedParam.tdPriority + 10);
/* create a new thread with priority = myPriority + 20 */
lid2 = newThreadAttr(K_MYACTOR, (KnPc) annex2, K_ACTIVE, 0x4000,
schedParam.tdClass, schedParam.tdPriority + 20);
K_MILLI_TO_TIMEVAL(&delay, 1100);
threadDelay(&delay); /* wait for a little more than one second */
fprintf(stderr, "Main thread [%d] before mutexGet\n", threadSelf( ));
mutexGet(&mutex);
fprintf(stderr, "Main thread [%d] after mutexGet\n", threadSelf( ));
K_MILLI_TO_TIMEVAL(&delay, 3000);
threadDelay(&delay); /* wait for 3 seconds */
}
--> rsh neon aps | grep serverCompute
0 25 serverCompute_s 0 N/A (=
server is active
--> neon mutex_u
started aid = 24
Entering first thread [9]
Entering second thread [14]
From compute: thread 14 in actor 24 starts computation
From compute: thread 9 in actor 24 starts computation
Main thread [13] before mutexGet
From compute: thread 9 in actor 24 ends computation
From compute: thread 14 in actor 24 ends computation
Main thread [13] after mutexGet
Second thread [14] after mutexRel
-->

9.4 Real-time mutexes


Real-time mutexes are provided by the RTMUTEX feature which is available within
release 4.0 of the system. They have as a side e ect that a thread which owns a
mutex and is therefore blocking another thread with a higher priority than its
own, takes the priority of this thread as a temporary priority. Real-time mutexes
are binary semaphores allowing synchronization of threads sharing objects. This
type of semaphore has two possible values, free or locked. The di erence from
260 Chapter 9. Synchronization
standard mutexes is that when a thread is blocking higher priority threads
because it owns one or several real-time mutexes, this thread executes at priority
either its own. or the highest priority of any thread currently blocked by a real-
time mutex it owns. If this thread becomes blocked on another real-time mutex,
this strategy applies also to the thread which owns the real-time mutex it wants
to acquire.
Real-time mutexes are allocated in user space and are of the KnRtMutex type.
A real-time mutex is initialized to the free value by a call to
#include <sync/chRtMutex.h>
int rtMutexInit(KnRtMutex *mutex);
A real-time mutex is acquired either by a blocking call (which is non abortable)
to
#include <sync/chRtMutex.h>
int rtMutexGet(KnRtMutex *mutex);
or by a non blocking call
#include <sync/chRtMutex.h>
int rtMutexTry(KnRtMutex *mutex);
With the rtMutexGet primitive, if the real-time mutex is locked, the calling
thread is blocked and the priority of the thread owning the real-time mutex is
set to the maximum of its own priority and the priority of the blocked thread.
A thread can free a real-time mutex by a call to
#include <sync/chRtMutex.h>
int rtMutexRel(KnRtMutex *mutex);
A side e ect of this type of call can be to lower the priority of the calling
thread if its priority was set by another thread when blocking during a call to
rtMutexGet.
Example
If real-time mutexes are used instead of normal mutexes, in the last example
(9.3.5), when the main thread blocks on the operation mutexGet, the thread
th2, which owns the mutex, takes as its new priority the value alpha which is
the priority of the thread it blocks. So, thread th1 which entered its loop is
preempted by th2. When thread th2 releases the mutex, it takes as its new
priority the value alpha + 20 it had before. The main thread is scheduled (it
gets the mutex) and when it nishes, thread th1 can nish its loop. At the end,
thread th2 can nish its work.
We therefore get the following results:
9.5. General semaphores 261
--> rsh neon arun cs -lM
started aid = 2
ChorusOS r4.0.0 Site 0 Time 1m 47
............. RTMUTEX .................
--> neon-n serverCompute_s.r&
[1] 4094
--> started aid = 2

--> neon rtMutex_u


started aid = 22
Entering first thread [8]
Entering second thread [7]
From compute: thread 7 in actor 22 starts computation
From compute: thread 8 in actor 22 starts computation
Main thread [9] before rtMutexGet (= main thread [12] is blocked
From compute: thread 7 in actor 22 ends computation
Main thread [9] after rtMutexGet (= main thread is elected
From compute: thread 8 in actor 22 ends computation
Second thread [7] after rtMutexRel (= thread th2 can terminate
-->

9.5 General semaphores


These are o ered by the SEM feature. General semaphores have integer values
and a queue is attached to each object. They correspond to the opaque KnSem
type prede ned in the <sync/chSem.h> standard header le. Semaphores are
allocated in the address space of actors and there is therefore no limit to the
number of semaphores an application may use. By mapping regions contain-
ing semaphores, di erent applications may share semaphores and thus, threads
belonging to di erent actors may be synchronized. The two classic atomic op-
erations P and V on semaphores are available: the rst one decrements, if
possible, the value of the semaphore by one and the second increments that
value by one.
9.5.1 Initializing a semaphore: semInit
Once it has been allocated, a semaphore has to be initialized by calling the
primitive
#include <sync/chSem.h>
int semInit(
KnSem *semaphore,
unsigned int value
);
This call initializes the semaphore pointed to by semaphore to the given value.
262 Chapter 9. Synchronization
When a semaphore is statically allocated it may be directly initialized by calling
the K KNSEM INITIALIZER macro as in the following sequence:
KnSem sem = K_KNSEM_INITIALIZER(5);
where the semaphore is allocated and is initialized to the value 5.
9.5.2 Waiting for a semaphore: semP
A call to the primitive
#include <sync/chSem.h>
int semP(
KnSem *semaphore,
KnTimeVal *waitLimit
);

decrements the value of the semaphore by one. If the new value is strictly
negative, the calling thread is blocked and queued in the queue corresponding to
the semaphore. The thread will remain blocked until a V operation is performed
on the semaphore, or until expiration of the period de ned by the value of
waitLimit. In particular, the K NOTIMEOUT value may be used for an abortable
in nite delay and the K NOTIMEOUT NOABORT value for a non abortable call.
9.5.3 Signaling a semaphore: semV
A call to the primitive
#include <sync/chSem.h>
int semV(KnSem *semaphore);

increments the value of the semaphore by one. If there are threads blocked
on the semaphore, one is awoken. The function returns K OK except if the
semaphore structure has been spoiled, in which case K EINVAL is returned.
9.5.4 Example
In the following example, which is a classic implementation of the producer-
consumer relationship, we demonstrate the possibility of synchronizing threads
belonging to di erent actors by mapping a region containing semaphores and
data in the address spaces of the actors.
The prodCons.h le contains the de nition of the structure related to the orga-
nization of the region shared by the producers and the consumers: the regions
contains two semaphores, a mutex and a circular bu er:
9.5. General semaphores 263
--> cat $CHORUS/include/prodCons.h
#include "utilities.h"
typedef struct {
KnSem full, /* # of used cells */
empty; /* # of unused cells */
KnMutex mutexRear, /* for filling a cell */
mutexFront; /* for consuming a cell */
int front, /* first used cell */
rear, /* first unused cell */
sizeArray; /* total # of cells */
int array[1];
} data;

The region is created and initialized by a repository actor whose main thread
executes the following command and terminates:
--> cat $CHORUS/sources/synchronization/semaphores/repository/repository.c
#include "prodCons.h"
data *sharedData;
main( ) {
KnCap actorCap;
KnRgnDesc region;
int result;
region.size = vmPageSize( );
region.options = K_ANYWHERE | K_WRITABLE | K_FILLZERO | K_NODEMAND;
result = rgnAllocate(K_MYACTOR, &region);
if(result != K_OK) {
fprintf(stderr, "error on regionAllocate: %s\n", strSysError(result));
exit(2);
}
/* sharedData -> sizeArray should be (vmPageSize( ) - n) / sizeof(int) */
/* where n = 2 * (sizeof(KnSem) + sizeof(KnMutex) + sizeof(int) */
sharedData = (data *) region.startAddr;
sharedData -> sizeArray = 4;
semInit(&sharedData -> empty, sharedData -> sizeArray);
semInit(&sharedData -> full, 0);
mutexInit(&sharedData -> mutexRear);
mutexInit(&sharedData -> mutexFront);
sharedData -> front = sharedData -> rear = 0;
actorSelf(&actorCap);
fprintCap(stdout, "Depositary actor", &actorCap);
printf("Data regions starts at %p\n", sharedData);
threadDelete(K_MYACTOR, K_MYSELF);
}

The producers execute the following application in the round robin scheduling
class: the capability of the repository and the address of the region in that actor
to be mapped to the producer's address space are passed as arguments:
264 Chapter 9. Synchronization
--> cat $CHORUS/sources/synchronization/semaphores/producer/producer.c
#include "prodCons.h"
#include <sched/chSched.h>
#include <sched/chRr.h>
data *sharedData;

produce( ) {
int ind;
compute(1); /* a computing step */
semP(&sharedData -> empty, K_NOTIMEOUT);
mutexGet(&sharedData -> mutexRear);
sharedData -> array[sharedData -> rear] = 1 + rand( ) % 1000;
fprintf(stderr, "%d has been produced by c_actor %d\n",
sharedData -> array[sharedData -> rear], agetId( ));
sharedData -> rear = (sharedData -> rear + 1) % sharedData -> sizeArray;
mutexRel(&sharedData -> mutexRear);
semV(&sharedData -> full);
}

main(int argc, char *argv[ ]) {


KnCap repositoryCap;
KnRgnDesc repositoryRegion, myRegion;
int result;
KnThreadDefaultSched schedAttr;
if(argc != 6) {
fprintf(stderr, "Five arguments when calling\n");
exit(2);
}
schedAttr.tdClass = K_SCHED_RR;
schedAttr.tdPriority = 140;
threadScheduler(K_MYACTOR, K_MYSELF, NULL, &schedAttr);
fprintf(stderr, "producer %d with thread %d\n", agetId( ), threadSelf( ));
readCap(argv + 1, &repositoryCap);
sscanf(argv[5], "%p", &repositoryRegion.startAddr);
myRegion.options = K_ANYWHERE | K_WRITABLE;
myRegion.size = vmPageSize( );
result = rgnMapFromActor(K_MYACTOR, &myRegion,
&repositoryCap, &repositoryRegion);
if(result != K_OK) {
fprintf(stderr, "error on regionMapFromActor: %s\n",
strSysError(result));
exit(2);
}
sharedData = (data *) myRegion.startAddr;
while(1)
produce( );
}
-->
9.5. General semaphores 265
Finally, the consumers correspond to the following application which will also
be executed in the round-robin scheduling class:
--> cat $CHORUS/sources/synchronization/semaphores/consumer/consumer.c
#include "prodCons.h"
#include <sched/chSched.h>
#include <sched/chRr.h>
data *sharedData;

consume( ) {
int ind, value;
semP(&sharedData -> full, K_NOTIMEOUT);
mutexGet(&sharedData -> mutexFront);
value = sharedData -> array[sharedData -> front];
fprintf(stderr, "%d has been read by c_actor %d\n", value, agetId( ));
sharedData -> front = (sharedData -> front + 1) % sharedData -> sizeArray;
mutexRel(&sharedData -> mutexFront);
semV(&sharedData -> empty);
compute(1); /* computation step */
}

main(int argc, char *argv[ ]) {


KnCap repositoryCap;
KnRgnDesc repositoryRegion, myRegion;
KnThreadDefaultSched schedAttr;
int result;
if(argc != 6) {
fprintf(stdout, "Five arguments when calling\n");
exit(2); }
schedAttr.tdClass = K_SCHED_RR;
schedAttr.tdPriority = 140;
threadScheduler(K_MYACTOR, K_MYSELF, NULL, &schedAttr);
fprintf(stderr, "consumer %d with thread %d\n", agetId( ), threadSelf( ));
readCap(argv + 1, &repositoryCap);
sscanf(argv[5], "%p", &repositoryRegion.startAddr);
myRegion.options = K_ANYWHERE | K_WRITABLE;
myRegion.size = vmPageSize( );
result = rgnMapFromActor(K_MYACTOR, &myRegion,
&repositoryCap, &repositoryRegion);
if(result != K_OK) {
fprintf(stderr, "error on regionMapFromActor: %s\n",
strSysError(result));
exit(2); }
sharedData = (data *) myRegion.startAddr;
while(1)
consume( );
}
-->
266 Chapter 9. Synchronization
An example of an execution with two consumers and two producers is now given:
--> rsh neon arun cs -lM
started aid = 22
ChorusOS r4.0.0 Site 0 Time 1h 13m 20
CORE SCHED_CLASS [ FIFO RR RT ] SEM .... MEM_PRM ...... VTIMER ....
--> rsh neon aps | grep serverCompute
0 2 serverCompute_s 0 N/A
--> neon-n repository_u &
[2] 4411
--> started aid = 22
Depositary actor: 2000001f 869da80a 16 0
Data regions starts at 0xffff4000

--> neon-n consumer_u 2000001f 869da80a 16 0 0xffff4000&\


neon-n producer_u 2000001f 869da80a 16 0 0xffff4000&\
neon-n producer_u 2000001f 869da80a 16 0 0xffff4000&\
neon-n consumer_u 2000001f 869da80a 16 0 0xffff4000&\
[3] 4415
[4] 4416
[5] 4418
[6] 4420
--> started aid = 23
consumer 23 with thread 9 (= rst consumer blocked on semP
started aid = 24
producer 24 with thread 7 (= rst producer
From compute: thread 7 in actor 24 starts computation
started aid = 25
producer 25 with thread 8 (= second producer
From compute: thread 8 in actor 25 starts computation
started aid = 26
consumer 26 with thread 10 (= second consumer blocked on semP
From compute: thread 8 in actor 25 ends computation
591 has been produced by c_actor 25 (= bu er = {591}
From compute: thread 8 in actor 25 starts computation
591 has been read by c_actor 23 (= bu er is empty
From compute: thread 9 in actor 23 starts computation
From compute: thread 7 in actor 24 ends computation
412 has been produced by c_actor 24 (= bu er = {412}
From compute: thread 7 in actor 24 starts computation
412 has been read by c_actor 26 (= bu er is empty
From compute: thread 10 in actor 26 starts computation
From compute: thread 8 in actor 25 ends computation
576 has been produced by c_actor 25 (= bu er = {576}
From compute: thread 8 in actor 25 starts computation
From compute: thread 9 in actor 23 ends computation
576 has been read by c_actor 23 (= bu er is empty
From compute: thread 9 in actor 23 starts computation
9.6. Event sets 267
From compute: thread 8 in actor 25 ends computation
85 has been produced by c_actor 25 (=bu er = {85}
From compute: thread 8 in actor 25 starts computation
From compute: thread 10 in actor 26 ends computation
85 has been read by c_actor 26 (=bu er is empty
From compute: thread 10 in actor 26 starts computation
From compute: thread 7 in actor 24 ends computation
273 has been produced by c_actor 24 (=bu er = {273}
From compute: thread 7 in actor 24 starts computation
From compute: thread 7 in actor 24 ends computation
175 has been produced by c_actor 24 (=bu er = {273,175}
From compute: thread 7 in actor 24 starts computation
From compute: thread 9 in actor 23 ends computation
273 has been read by c_actor 23 (=bu er = {175}
From compute: thread 9 in actor 23 starts computation
From compute: thread 8 in actor 25 ends computation
782 has been produced by c_actor 25 (=bu er = {175,782}
From compute: thread 8 in actor 25 starts computation
From compute: thread 10 in actor 26 ends computation
175 has been read by c_actor 26 (=bu er = {782}
...... [ ]
and so on .........
9.6 Event sets
9.6.1 Overview
The EVENT feature exports an API to synchronize threads with events. An
event set is associated to the KnEventSet opaque type and these types of
objects have to be allocated by the actor in its address space. The set itself is
coded on an integer: each bit of the integer corresponds to one event. Thus, it
is possible to code 8*sizeof(int) events: each event is numbered by an integer
in the range [0::8  sizeof (int) 1]. When the bit associated to one event is 1,
the event is said POSTED (UNPOSTED otherwise).
Using an event set rst requires the allocation of a KnEventSet object. This
allocation can be static using a direct de nition KnEventSet event. An event
set can also be dynamically allocated, allowing the synchronization of threads
belonging to di erent actors by mapping the corresponding region in the address
space of their respective actors.
9.6.2 Initializing an event set: eventInit
Once allocated, an event set must be initialized before being used by calling the
primitive
#include <sync/chEvent.h>
int eventInit(KnEventSet *eventSet);
268 Chapter 9. Synchronization
After that operation every event in the set is therefore UNPOSTED.
The function returns K OK upon success. The K EFAULT value is returned if
eventSet points outside the address space of the actor.
When it is statically allocated, an event set may be directly initialized (every
event is UNPOSTED by using the K KNEVENTSET INITIALIZER macro as in the
following sequence:
KnEventSet eventSet = K KNEVENTSET INITIALIZER( );

9.6.3 Posting an event: eventPost


A call to the function
#include <sync/chEvent.h>
int eventPost(
KnEventSet *eventSet,
unsigned int mask
);

is a request for posting, in the event set pointed to by eventSet, all the events
corresponding to a bit set to 1 in the mask.
For instance, if mask has the decimal value 194 which corresponds to hexadec-
imal 0xc2 or to the binary value 11000010, the events 1, 6 et 7 are posted.
This value corresponds to (1<<1)+(1<<6)+(1<<7). The call has no e ect for
the events of the set which are already in the POSTED state.
The function returns K OK upon success and a negative value otherwise (K EFAULT
if eventSet points outside the address space and K EINVAL if it has not been
correctly initialized).

9.6.4 Waiting for events: eventWait


A call to the function
#include <sync/chEvent.h>
int eventWait(
KnEventSet *eventSet,
unsigned int inMask,
unsigned int option,
unsigned int *outMask,
KnTimeVal *waitLimit
);
9.6. Event sets 269
is a request for waiting on one or more events to be posted in the given event
set. In this type of call:
 the inMask parameter is a mask of awaited events. If this argument is NULL
no event is awaited and the calling thread returns immediately: it may be used
for getting the current status of the event set within outMask;
 the option integer argument may have the following values:
- K_EVENT_OR: the calling thread is blocked if no event matching the mask
de ned by inMask has been posted; that is, if all events in the event set
are in the UNPOSTED state. Thus, the calling thread will be awoken as
soon as one of the events becomes posted;
- K_EVENT_AND : the calling thread is blocked if at least one event de ned
by inMask is in the state UNPOSTED. The calling thread will be awoken
when all awaited events become posted;
 as usual, the value de ned by waitLimit allows a real blocking call: it may be
K_NOTIMEOUT or de ne a speci c timeout;
 when the thread is awoken, outMask provides the state of the event ag set.
If the awaited condition is satis ed, the call may return immediately.
It is also important to note that when the events blocking a thread are posted,
the thread will be awoken: execution will resume after the call.

9.6.5 Clearing events: eventClear


It is important to note that when a thread waiting for events is awoken, the
events are not consumed: they remain in the POSTED state.
The transition of events to the UNPOSTED state must be explicitly requested
by a call to the function
#include <sync/chEvent.h>
int eventClear(
KnEventSet *eventSet,
unsigned int mask
);

After this call, the events of the set pointed to by eventSet corresponding to
bits of the mask whose value is 1 are in the UNPOSTED state (the call has no
e ect on events which were already in the UNPOSTED state).
270 Chapter 9. Synchronization
9.6.6 Example: a synchronization barrier's implementa-
tion
In the following example, we use the service provided by event sets to imple-
ment a barrier synchronization of n threads cooperating in an application [see
Andrews]. All threads belong to the same actor and each thread involved in
the application is identi ed by an index: the main thread that creates the other
threads is identi ed by 0, the other threads are identi ed by 1, . . . , n-1 if
there are n threads. A thread arriving at the barrier will post, in the event
set arrived the event corresponding to its identi cation in the application. So
when all threads have reached the barrier, n events will have been posted. In
a rst approach, the "rendez-vous" of the n threads at the barrier is achieved
symmetrically by calling eventWait in each thread with the {0,1, ...,n-1}
value.
In the following example, several threads belonging to the same actor execute
the same annex function whose argument is the index of the thread. Each
thread executes the following operations:
- it rst enters a computation phase preceding the barrier corresponding
to the compute function. The CPU time consumed when executing this
function is computed randomly and may vary from one thread to another
and even from one call to another for a given thread;
- after this computation, it signals its arrival at the barrier by posting the
event corresponding to its number in the arrived event set that should
have been initialized previously;
- it waits for all other threads to reach the barrier: fullMask contains the
set of all events between 0 and n-1;
- it terminates.
We now give the code for the onceBarrier application, where the main thread
uses the newThreadArg function (see 8.5.1.2.d) to create a new thread executing
a function with an integer argument. When the command is called, the total
number of threads participating in the rendez-vous (including the main thread)
is passed as rst argument and if more arguments are given, the round robin
scheduling policy is selected:
--> cat $CHORUS/sources/synchronization/events/onceBarrier/onceBarrier.c
#include "utilities.h"
#include <sync/chEvent.h>
KnCap actorCap;
int numberOfThreads; /* # of participating threads */
KnEventSet arrived; /* for signaling threads' arrival */
unsigned int fullMask;
9.6. Event sets 271
/* function executed by the thread: ident is the thread's index */
void annex(int ident) {
unsigned int ind, myMask;
fprintf(stderr, "entering thread %d\n", ident);
/* build the mask corresponding to the thread's index */
myMask = 1 << ident;
compute(1); /* compute step before the barrier */
/* thread arrives to the barrier */
fprintf(stderr, "Thread %d arrived at barrier\n", ident);
/* the corresponding event is posted to signal its arrival at barrier */
eventPost(&arrived, myMask);
/* the thread waits for all threads to signal their arrival at barrier */
eventWait(&arrived, fullMask, K_EVENT_AND, NULL, K_NOTIMEOUT);
fprintf(stderr, "Thread %d after the barrier\n", ident);
threadDelete(K_MYACTOR, K_MYSELF);
}
main(int argc, char *argv[ ]) {
int result, ind;
unsigned int myMask;
KnThreadLid threadLi;
KnThreadDefaultSched schedAttr;
if (argc == 1) {
fprintf(stderr, "bad usage\n");
exit(1); }
if (argc > 1) {
schedAttr.tdClass = K_SCHED_RR;
schedAttr.tdPriority = 140;
threadScheduler(K_MYACTOR, K_MYSELF, NULL, &schedAttr); }
/* the total number of threads is passed as first argument */
numberOfThreads = atoi(argv[1]);
eventInit(&arrived); /* event set is initialized */
/* the mask corresponding to all threads is built */
fullMask = 0;
for(ind = 0; ind < numberOfThreads; ind ++)
fullMask += (1 << ind);
actorSelf(&actorCap);
srand(actorCap.key.keyHead + actorCap.key.keyTail); /* initialize random */
/* loop to create the new threads */
for(ind = 1; ind < numberOfThreads; ind ++) {
/* new thread created with newThreadArg */
threadLi = newThreadArg((KnPc) annex, K_ACTIVE, NULL, 1, ind);
if (threadLi == -1) {
fprintf(stderr, "error on newThreadArg\n");
exit(2); }
fprintf(stderr, "thread %d [application's id %d] is created\n",
threadLi, ind); }
/* the main thread is participating to the rendez-vous */
annex(0);
}
272 Chapter 9. Synchronization
The command is rst executed to achieve a rendez-vous of three threads (value
3 of the argument) within the fo scheduling policy. When all threads have ter-
minated, the actor has to be killed (the actor still exists but no longer contains
threads):
--> rsh neon arun cs -lM
started aid = 22
ChorusOS r4.0.0 Site 0 Time 2h 20m 0
CORE SCHED_CLASS [ FIFO RR RT ] VTIMER ..... EVENT ......
--> rsh neon aps | grep serverCompute
0 2 serverCompute_s 0 N/A
--> neon-n onceBarrier_u 3& (=3 is the number of threads
[2] 4580
--> started aid = 22
thread 7 [application's id 1] is created
thread 8 [application's id 2] is created
entering thread 0
From compute: thread 9 in actor 22 starts computation
entering thread 1
From compute: thread 7 in actor 22 starts computation
entering thread 2
From compute: thread 8 in actor 22 starts computation
From compute: thread 8 in actor 22 ends computation
Thread 2 arrived at barrier
From compute: thread 9 in actor 22 ends computation
Thread 0 arrived at barrier
From compute: thread 7 in actor 22 ends computation
Thread 1 arrived at barrier
Thread 1 after the barrier
Thread 2 after the barrier
Thread 0 after the barrier

--> rsh neon arun cs | grep onceBarr


started aid = 23
20000025 869da80a 00000016 00000103 0022 USER STARTED 000 onceBarrier_u
--> rsh neon akill 22 (= actor without threads is killed
-->

Threads arrive at the barrier in the order in which they were created (0 identi es
the main thread in the application). We can observe that a thread is blocked
until all other threads have reached the barrier. When the last thread reaches
the barrier, it posts its event. Thus, when it calls eventWait, all threads have
posted their events and the awaited condition is satis ed; the thread is not
blocked and can terminate its execution (it is the rst thread which signals that
it has passed the barrier). Other threads are then awoken and resume execution
in the order in which they were blocked (0 and then 1).
9.6. Event sets 273
The same application has been executed within the round robin scheduling
policy: as the computation time is di erent for every thread and is signi cant,
time slicing applies e ectively and threads arrive at the barrier in a random
order (here 2, 1 and 0):
--> neon-n onceBarrier_u 3 RR & (= 3 is the number of threads
[2] 4614
--> started aid = 22
thread 9 [application's id 1] is created
thread 7 [application's id 2] is created
entering thread 0
From compute: thread 8 in actor 22 starts computation
entering thread 1
From compute: thread 9 in actor 22 starts computation
entering thread 2
From compute: thread 7 in actor 22 starts computation
From compute: thread 7 in actor 22 ends computation
Thread 2 arrived at barrier
From compute: thread 9 in actor 22 ends computation
Thread 1 arrived at barrier
From compute: thread 8 in actor 22 ends computation
Thread 0 arrived at barrier
Thread 0 after the barrier
Thread 2 after the barrier
Thread 1 after the barrier
--> rsh neon akill 22 (=actor is killed
-->

What happens if, after this barrier, the di erent threads have to enter a new
computation cycle and have to meet again at the same barrier? The event set
has to be cleared and the diculty is to clear it properly. An event can be
cleared only when all threads waiting on this event have passed the barrier, and
all events have to be cleared before any thread in the set reaches the barrier
again.
A symmetrical approach cannot be used. We give below the complete source
code of a barrier application where the rst parameter is the total number of
threads and the second one de nes the number of steps that each thread will
perform (this value is the number of times the barrier will be passed by each
thread). If other parameters are given, they correspond to a request for round
robin scheduling.
--> cat $CHORUS/sources/synchronization/events/barrier/barrier.c
#include "utilities.h"
#include <sync/chEvent.h>
int numberOfThreads, /* # of participating threads */
numberOfSteps; /* # of iterations (# of rendez-vous) */
274 Chapter 9. Synchronization
KnEventSet arrived, /* for signaling threads arrival */
nextStep; /* for allowing threads continuation */

void annex(int ident) {


unsigned int ind, myMask;
fprintf(stderr, "entering thread %d\n", ident);
myMask = 1 << ident;
srand(rand( ) + ident);
for(ind = 0; ind < numberOfSteps; ind ++) {
compute(0);
fprintf(stderr, "Thread %d arrived step %d\n", ident, ind);
/* corresponding event is posted in arrived event set */
eventPost(&arrived, myMask);
eventWait(&nextStep, myMask, K_EVENT_AND, NULL, K_NOTIMEOUT);
fprintf(stderr, "Thread %d continues\n", ident);
eventClear(&nextStep, myMask);
}
fprintf(stderr, "Thread %d terminating\n", ident);
threadDelete(K_MYACTOR, K_MYSELF);
}

main(int argc, char *argv[ ]) {


int result, ind;
unsigned int myMask, fullMask = 0;
KnThreadLid threadLi;
KnThreadDefaultSched schedAttr;
if (argc < 3) {
fprintf(stderr, "bad usage\n");
exit(1); }
if (argc > 3) {
schedAttr.tdClass = K_SCHED_RR;
schedAttr.tdPriority = 140;
threadScheduler(K_MYACTOR, K_MYSELF, NULL, &schedAttr); }
numberOfThreads = atoi(argv[1]);
numberOfSteps = atoi(argv[2]);
eventInit(&arrived);
eventInit(&nextStep);
for(ind = 1; ind < numberOfThreads; ind ++)
fullMask += (1 << ind);
for(ind = 1; ind < numberOfThreads; ind ++) {
/* new thread created with newThreadArg */
threadLi = newThreadArg((KnPc) annex, K_ACTIVE, NULL, 1, ind);
if (threadLi == -1) {
fprintf(stderr, "error on newThreadArg\n");
exit(2);
}
fprintf(stderr, "thread %d [application's id %d] is created\n",
threadLi, ind);
}
9.6. Event sets 275
for(ind = 0; ind < numberOfSteps; ind ++) {
compute(0);
fprintf(stderr, "Thread 0 arrived step %d\n", ind);
/* waiting for every other thread to have posted its event */
eventWait(&arrived, fullMask, K_EVENT_AND, NULL, K_NOTIMEOUT);
/* all events are cleared in arrived event set */
eventClear(&arrived, fullMask);
fprintf(stderr, "----------------------------\n");
/* all events are posted in nextStep event set */
eventPost(&nextStep, fullMask);
fprintf(stderr, "Thread %d continues\n", 0);
}
fprintf(stderr, "Thread %d terminating\n", 0);
}
We rst execute the application within the FIFO scheduling policy and we can
observe the sequential, synchronized executions of the di erent steps. Execution
is deterministic: all executions produce the same results:
--> neon barrier_u 3 3
started aid = 22
thread 9 [application's id 1] is created
thread 7 [application's id 2] is created
Thread 0 arrived step 0
entering thread 1
Thread 1 arrived step 0
entering thread 2
Thread 2 arrived step 0
----------------------------
Thread 0 continues
Thread 0 arrived step 1
Thread 1 continues
Thread 1 arrived step 1
Thread 2 continues
Thread 2 arrived step 1
----------------------------
Thread 0 continues
Thread 0 arrived step 2
Thread 1 continues
Thread 1 arrived step 2
Thread 2 continues
Thread 2 arrived step 2
----------------------------
Thread 0 continues
Thread 0 terminating
-->
As the main thread terminates with an implicit exit call, the actor is deleted;
thus, the annex threads are deleted.
276 Chapter 9. Synchronization
If the command is executed within the round robin scheduling policy for the
threads, we observe the "random" order of arrivals at the barrier but a consistent
execution of the di erent steps:
--> neon barrier_u 3 3 RR
started aid = 22
thread 9 [application's id 1] is created
thread 8 [application's id 2] is created
entering thread 1
entering thread 2
Thread 0 arrived step 0
Thread 2 arrived step 0
Thread 1 arrived step 0
----------------------------
Thread 0 continues
Thread 2 continues
Thread 1 continues
Thread 1 arrived step 1
Thread 0 arrived step 1
Thread 2 arrived step 1
----------------------------
Thread 0 continues
Thread 1 continues
Thread 2 continues
Thread 0 arrived step 2
Thread 1 arrived step 2
Thread 2 arrived step 2
----------------------------
Thread 0 continues
Thread 0 terminating
-->

In this type of environment, execution is not deterministic: di erent executions


may lead to di erent results.

9.7 Posix synchronization mechanism


9.7.1 Overview
Di erent services are provided by Posix for synchronizing thread execution:
 waiting for thread termination (pthread join);
 Posix mutexes;
 conditions;
 yielding processor.
9.7. Posix synchronization mechanism 277
9.7.2 Waiting for thread termination
When calling the function
#include <pthread.h>
int pthread join(
pthread t thread,
void **status
);

a thread suspends its execution until the target thread terminates, unless it
already terminated or it was created in the PTHREAD CREATE DETACHED state.

9.7.3 Posix mutexes


9.7.3.1 Attributes of Posix mutexes
Mutexes have the pthread mutex t type and a mutex has two possible states:
it may be locked or unlocked. A set of attributes is attached to a mutex: this
set corresponds to the pthread mutexattr t type.
A mutex that has been previously allocated is initialized by calling the function

#include <pthread.h>
int pthread mutex init(
pthread mutex t *mutex,
const pthread mutexattr t *mutexAttr
);

If the mutexAttr value is NULL, default mutex attributes are used. If the mutex
is statically allocated, it may be initialized with the default attributes by using
the PTHREAD MUTEX INITIALIZER macro as in the following:
pthread mutex t mutex = PTHREAD MUTEX INITIALIZER;
A mutex may be destroyed by calling the function
#include <pthread.h>
int pthread mutex destroy(pthread mutex t *mutex);

After the call the mutex has an illegal value. This type of call should not be
applied to a locked mutex.
278 Chapter 9. Synchronization
9.7.3.2 Locking a Posix mutex
A mutex may be locked by calling
#include <pthread.h>
int pthread mutex lock(pthread mutex t *mutex);
If the mutex is already locked, the calling thread is blocked. If a thread tries to
lock a mutes that it already locked, a deadlock will occur.
Locking a mutex may also be achieved by a non blocking call:
#include <pthread.h>
int pthread mutex trylock(pthread mutex t *mutex);
The function returns -1 and errno has the value EBUSY.
9.7.3.3 Unlocking a Posix mutex
A thread that previously locked a mutex may release it by calling
#include <pthread.h>
int pthread mutex unlock(pthread mutex t *mutex);
The thread that has been waiting the longest time (if any) for the mutex will
be awakened and will return from its pthread mutex lock call.
9.7.4 Posix condition variables
Condition variables correspond to the pthread cond t type and their use is
associated with Posix mutexes. The principles of use are as follows:
 given a var variable of any type, a thread may wait before resuming execu-
tion for a condition to be satis ed by the value of this variable (this condition
corresponds to the true value of a boolean expression);
 a var mutex mutex and a var cond condition variable are associated to this
variable. The mutex is used for protecting operations on var and the condition
variable is used for transmitting its change of state.
9.7.4.1 Initializing a condition variable
A pthread cond t object encapsulates attributes of a variable condition. This
type of variable is initialized by calling the function
#include <pthread.h>
int pthread cond init(
pthread cond t *cond,
pthread condattr t *attr
);
9.7. Posix synchronization mechanism 279
When attr is NULL, the default value is used. A statically allocated condition
variable may be initialized to the default value as in the following sequence:
pthread cond t cond = PTHREAD COND INITIALIZER;
It is possible to destroy a condition variable if no thread is currently blocked on
it by calling the primitive
#include <pthread.h>
int pthread cond destroy(pthread cond t *cond);

9.7.4.2 Waiting on a condition variable


Once it has acquired the corresponding mutex, a thread may block on a variable
condition by calling the function
#include <pthread.h>
int pthread cond wait(
pthread cond t *condition,
pthread mutex t *mutex
);

The e ects of this call are the following:


- the mutex is released;
- the calling thread is blocked on the condition;
- when the calling thread awakes, the mutex will be re-acquired by the
thread.

9.7.4.3 Signaling a condition variable


A condition variable is signaled by calling the primitive
#include <pthread.h>
int pthread cond signal(pthread cond t *cond);

This awakens the thread, if any, that has been blocked on the condition for the
longest time. If no thread is currently blocked on the condition variable, the
call has no e ect.
It is also possible to awaken all threads that are currently blocked on a condition
by calling
#include <pthread.h>
int pthread cond broadcast(pthread cond t *cond);
280 Chapter 9. Synchronization
9.7.4.4 General outline
We just give here the general outline for using condition variables.
a) from the waiting thread point of view:
/* waiting for var to satisfy a condition */
type var;
pthread cond t var cond;
pthread mutex t var mutex;
. . .
. . .
pthread mutex lock(&var mutex);
while(!condition(var)){
/* waiting for var cond to be signaled */
pthread cond wait(&var cond, &var mutex);
}
/* we can use the variable and resume execution */
.
.
/* var mutex is released */
pthread mutex unlock(&var mutex);

b) from the signaling thread point of view:


. . .
. . .
/* changing the value of var under protection */
pthread mutex lock(&var mutex);
var = new value;
pthread mutex unlock(&var mutex);
. . .
. . .
pthread cond signal(&var cond);
. . .
. . .
Chapter 10
Private data
10.1 Introduction
Threads belonging to a given actor share the address space of that actor and
supervisor actors share the supervisor address space. In some applications it
may be helpful to de ne data speci c to one entity.
A rst typical example concerns the errno variable which is used in a classical
mono-threaded POSIX system to specify an error when a system call fails (in
ChorusOS it is used for that purpose when the actor manager is invoked). The
header le <errno.h> is generally used to declare this variable in an application
(it contains the corresponding declaration extern int errno;). In a multi-
threaded environment, each thread should own its private errno variable to
avoid a bad interpretation of the error values.
Another example is the reuse of an application developed in a mono-threaded
environment in a multi-threaded one. This application could be similar to the
following which returns the value of a counter:
static int counter = 0;
int counterValue( ) { return ++counter; }
When implemented in a multi-threaded actor, it could be desirable that:
 the value returned by a call corresponds to a counter speci c to the executing
thread: thus, an array of values indexed by a thread identi er is needed;
 the values are usable concurrently by di erent routines;
 data corresponding to a given thread are cleaned up when the thread termi-
nates.
The executive implements a high-level interface allowing the management of
private per-thread data within the address space of an actor, and per-actor
private-data service for supervisor actors. The POSIX-THREADS API also
281
282 Chapter 10. Private data
provides a service for de ning thread speci c data.
The PD actor is in charge of managing private data:
--> rsh neon arun cs | grep PD
started aid = 25
20000011 869da80a 00000011 00000001 0017 SUP STARTED 000 PD
--> rsh neon arun cs -la 17
started aid = 25
ChorusOS r4.0.0 Site 0 Time 2d 13h 6m 26
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000011 869da80a 00000011 00000001 0017 SUP STARTED 000 PD
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
20000011 869da80a def no fac778 0 20000011 869da80a
START SIZE OFFSET ALLOC OPTIONS
40036000 00002000 00000000 00002000 EX SU
40038000 00001000 00000000 00001000 WR SU
-->
To provide the private data services, it relies in turn on the software register
mechanism supported by the kernel (see 8.11). It provides an extension of the
software mechanism so that actors' libraries may bene t from them without
competing for these registers.
The executive exports di erent functions which all return the value 0 upon
success. A positive error code is returned when an error occurs in a call: the
error values are listed in appendix A.

10.2 Standard services


10.2.1 The errno variable
Within the private data service, a private errno word is associated with every
thread. The address of this may be obtained by calling the function
#include <pd/chPd.h>
int *ptdErrnoAddr(void);

The standard header le errno.h used in the ChorusOS contains the following
macros:
#define __errno ptdErrnoAddr
#define errno (*(__errno( )))
which allow each thread to access its private errno word. The following example
shows that property:
10.2. Standard services 283
-1-> cat $CHORUS/sources/privateData/errno/errno.c
#include "utilities.h"
#include "fcntl.h"
#include "errno.h"

void annex(int i) {
printf("annex %d: address of errno=%p errno=%d\n",
i, &errno, errno);
threadDelay(K_NOTIMEOUT); /* yield the processor */
threadDelete(K_MYACTOR, K_MYSELF);
}

main( ) {
int descrpt;
KnCap actorCap;
actorSelf(&actorCap);
fprintCap(stdout, "actor", &actorCap);
printf("main thread is %d\n", threadSelf( ));
threadDelay(K_NOTIMEOUT); /* yield the processor */
descrpt = open("/file", O_RDONLY); /* non existing file */
if(descrpt == -1)
printf("main: address of errno=%p errno=%d\n", &errno, errno);
newThreadArg((KnPc) annex, K_ACTIVE, "annex", 1, 1);
newThreadArg((KnPc) annex, K_ACTIVE, "annex", 1, 2);
threadDelay(K_NOTIMEOUT); /* yield the processor */
}
-1->
-2-> neon errno_u
neon errno_u
started aid = 25
actor: 20000143 869da80a 19 0
main thread is 9

-1-> rsh neon arun cs -la 25


started aid = 24
ChorusOS r4.0.0 Site 0 Time 2d 13h 30m 42
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000143 869da80a 00000019 00000000 0025 USER STARTED 001 errno_u
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0009 140 00000000 00000000 ff6018 0- 1- 0 main
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
20000143 869da80a def no fac0ac 0 20000143 869da80a
START SIZE OFFSET ALLOC OPTIONS
fffee000 00006000 00000000 00006000 WR
ffff5000 00001000 00000000 00001000 WR
ffff6000 00009000 00000000 00009000 EX
284 Chapter 10. Private data
-1-> neon threadAbort_u 20000143 869da80a 19 0 9
started aid = 24
-1->

main: address of errno=0xffff5788 errno=2


annex 1: address of errno=0xffff4008 errno=0
annex 2: address of errno=0xffff4108 errno=0

-1-> rsh neon arun cs -la 25


started aid = 24
ChorusOS r4.0.0 Site 0 Time 2d 13h 35m 9
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000143 869da80a 00000019 00000000 0025 USER STARTED 003 errno_u
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0007 140 00000000 00000000 ff6380 0- 1- 0 annex
0008 140 00000000 00000000 ff61cc 0- 1- 0 annex
0009 140 00000000 00000000 ff6018 0- 1- 0 main
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
20000143 869da80a def no fac0ac 0 20000143 869da80a
START SIZE OFFSET ALLOC OPTIONS
fffe6000 00010000 00000000 00010000 WR
ffff6000 00009000 00000000 00009000 EX
-1-> neon threadAbort_u 20000143 869da80a 19 0 9
started aid = 24

-2->
These results show that the errno variable has di erent addresses in the dif-
ferent threads: these addresses lie within the rst data region allocated to the
actor and are accessed through the per-thread data service.
10.2.2 Threads' local identi ers
A call to the function
#include <pd/chPd.h>
int ptdThreadId(void);

returns the unique thread identi er for the thread (it is the same as the thread
local identi er returned by a call to threadSelf).
10.3 The private data keys
Like the corresponding POSIX service, the de nition of threads' or supervisor
actors' private data is based on the keys concept: this key has the type pdKey
(de ned as unsigned int). The keys are opaque indexes which are dynamically
allocated and are used to locate the thread-speci c data.
10.4. Chorus per-thread private data 285
At most PTD KEYS MAX(default value is 62) keys may be de ned. The values
0 and are associated to errno (PTD ERRNO) and the local thread identi er
1
( ).
PTD THREADID

10.4 Chorus per-thread private data


This service is provided for all threads (that is, belonging either to user or
supervisor actors).

10.4.1 Creating and deleting a thread speci c data key


A key that will be visible to all threads belonging to a given actor is created by
calling the primitive
#include <pd/chPd.h>
int ptdKeyCreate(
PdKey *privateKey,
KnPdHdl destructor
);

This call creates a new key whose value is returned at the privateKey address in
the calling actor's address space. This value is an opaque index that is used for
locating thread-speci c data. The same key may be used for di erent threads,
but the values that are associated to a key are maintained on a per-thread basis.
This value will persist throughout the life of a thread.
Once a key has been created, the NULL value is associated to the new key for
every active thread. If a new thread is created in an actor, the NULL value will
also be bound to the di erent existing keys.
The optional destructor parameter allows a destructor function to be associated
to a key. Then, when a thread exits, if the value associated to the key is
not NULL, the function will be called with the current value associated to the
key as its unique parameter. For user level threads, the destructor function
will be invoked if the thread calls ptdThreadDelete. For supervisor threads,
the destructor function will be invoked at thread termination (threadDelete,
actorDelete, . . . ).
The function returns 0 (and *privateKey is initialized) if the call is successful
and a positive error value otherwise:
return value error type
PD ENOKEY no more keys available
PD ENOMEM not enough memory to create the key
PD ESERVER the PD manager is not accessible
286 Chapter 10. Private data
A thread-speci c key previously allocated by calling ptdKeyCreate may be
deallocated by calling the primitive
#include <pd/chPd.h>
int ptdKeyDelete(PdKey key);

The values associated with the key need not to be NULL but it is the respon-
sibility of the application to perform cleanup operations either before or after
calling ptdKeyDelete. No destructor function is called by ptdKeyDelete.
Once a key has been deleted, it should not be used any more: behavior is un-
de ned if it is used.
The function returns 0 if it succeeds and PD EINVAL if key is not a valid value
or PD ESERVER if the PD server is unreachable.

10.4.2 Setting a thread-speci c value


A value may be associated to the current thread for an existing key by calling
the primitive
#include <pd/chPd.h>
int ptdSet(
PdKey key,
const void *value
);

This type of value is typically a pointer to a block of data that is dynamically


allocated by the thread. If the function is called from a destructor function a
storage leak may result.
The function returns 0 upon success. Otherwise, it returns a positive error value
(PD EINVAL, PD ENOMEM or PD ESERVER).
A thread may also set a value associated to a key for an existing thread by
calling the primitive
#include <pd/chPd.h>
int ptdRemoteSet(
KnCap *actorCap,
KnThreadLid threadLi,
PdKey key,
void *value
);
10.4. Chorus per-thread private data 287
This type of call associates the given value to the thread locally identi ed by
threadLi in the actor whose capability is pointed to by actorCap for the speci ed
key (which must have been previously created in the target actor). When this
primitive is called in user mode, actorCap must be the current actor.
A call returns 0 when it successful. Otherwise, it returns a positive error value
(PD EINVAL, PD ENOMEM or PD ESERVER).

10.4.3 Getting a thread speci c value


#include <pd/chPd.h>
void *ptdGet(PdKey key);

If no value is currently associated to the key in the current thread, the func-
tion returns NULL. If the value of key has not been obtained as result of a
ptdKeyCreate call or if it has been deleted by calling ptdKeyDelete, the e ect
of the call is unde ned.
A thread may also get the value associated to a key in another thread by calling
the primitive
#include <pd/chPd.h>
int ptdRemoteGet(
KnCap *actorCap,
KnThreadLid threadLi,
PdKey key,
void **value
);

The value associated to the key in the thread identi ed by threadLi in the actor
designated by actorCap is returned at *value address. When it is called in user
mode, actorCap must designate the current actor.
The function returns 0 if successful. Otherwise, it returns a positive error code
(PD EINVAL or PD ESERVER).

10.4.4 Deleting all thread speci c values


A thread may delete all the values associated to itself within the valid keys of
its actor by calling the primitive
#include <pd/chPd.h>
void ptdThreadDelete(void);
288 Chapter 10. Private data
For each value other than NULL, the corresponding destructor is called before
clearing the value. When it is called in user mode, this function should be called
before thread deletion to enable the destructor function calls. In supervisor
mode this call has no e ect.
10.4.5 Example
We illustrate the use of per-thread private data by implementing the counter
service that we presented in the introduction. This service may be accessed by
any thread by a call to the counterValue function. When a thread calls the
function:
 if the service has not yet been called by a thread of its actor, the key has
to be created for the actor. This check is performed by testing the value of
the initialized static variable: the test of the value and the update of this
variable are protected by a mutex;
 if the thread is calling the service for the rst time, the value returned by
ptdGet is NULL. An integer is allocated and initialized to 0;
 the service returns the current value of the thread speci c data associated to
the counter key and increments it.
The corresponding code is the following:
--> cat $CHORUS/sources/privateData/counter/counterValue.c
#include <stdio.h>
#include <stdlib.h>
#include <chorus.h>
#include <pd/chPd.h>
#include <sync/chMutex.h>
PdKey counterKey;
KnMutex mutex = K_KNMUTEX_INITIALIZER;
int initialized = 0; /* to test if the key has been created */

int counterValue( ) {
int *cnt;
mutexGet(&mutex);
if(initialized == 0) { /* the key has not been created yet */
/* the key is created */
if (ptdKeyCreate(&counterKey, free) != 0) {
/* the key has not been created: the actor exits */
mutexRel(&mutex);
exit(1);
}
fprintf(stderr, "the key (%d) has been created in actor %d\n",
counterKey, agetId( ));
initialized = 1;
}
10.4. Chorus per-thread private data 289
mutexRel(&mutex);
/* value associated to the key */
cnt = ptdGet(counterKey);
if (cnt == NULL) {
/* if the value is NULL, a counter is allocated */
cnt = (int *) malloc(sizeof (int));
/* the counter is initialized to 0 */
*cnt = 0;
/* the counter is associated to the key */
if(ptdSet(counterKey, cnt) != 0)
threadDelete(K_MYACTOR, K_MYSELF);
}
return (*cnt) ++;
}
-->

We now present an example of an application using this service. The application


corresponds to an actor creating three new threads: these threads and the main
thread simply call counterValue as long as the return value is not 2 (thus, each
thread should call counterValue twice):
--> cat $CHORUS/sources/privateData/counter/useCounter.c
#include "utilities.h"
int counterValue( );
KnCap actorCap;

void annex(int num) {


KnTimeVal delay;
int counter;
fprintf(stderr, "In thread %d in actor %d\n", num, agetId( ));
srand(actorCap.key.keyHead + actorCap.key.keyTail + num);
while ((counter = counterValue( )) != 2) {
fprintf(stderr, "actor %d thread %d: counter = %d\n",
agetId( ), num, counter);
/* a random value of the delay */
K_MILLI_TO_TIMEVAL(&delay, 300 * (1 + rand( ) % 5));
threadDelay(&delay);
}
threadDelete(K_MYACTOR, K_MYSELF);
}

main( ) {
int ind;
for(ind = 1; ind < 4; ind ++)
newThreadArg((KnPc) annex, K_ACTIVE, "annex", 1, ind);
annex(0);
}
290 Chapter 10. Private data
Two actors corresponding to the useCounter application are created. We can
observe the creation of the key in the actors, the execution of the di erent
threads and the evolution of the private counter of each thread:
--> neon-n useCounter_u & neon-n useCounter_u &
[1] 3256
[2] 3257
--> started aid = 25
In thread 0 in actor 25
the key (3) has been created in actor 25
actor 25 thread 0: counter = 0
In thread 1 in actor 25
actor 25 thread 1: counter = 0
In thread 2 in actor 25
actor 25 thread 2: counter = 0
In thread 3 in actor 25
actor 25 thread 3: counter = 0
started aid = 24
In thread 0 in actor 24
the key (3) has been created in actor 24
actor 24 thread 0: counter = 0
In thread 1 in actor 24
actor 24 thread 1: counter = 0
In thread 2 in actor 24
actor 24 thread 2: counter = 0
In thread 3 in actor 24
actor 24 thread 3: counter = 0
actor 25 thread 1: counter = 1
actor 24 thread 1: counter = 1
actor 25 thread 3: counter = 1
actor 25 thread 0: counter = 1
actor 24 thread 3: counter = 1
actor 24 thread 0: counter = 1
actor 25 thread 2: counter = 1
actor 24 thread 2: counter = 1
-->

Finally when all threads are terminated, actors have no more threads and are
killed:
--> rsh neon arun cs | grep useCounter
started aid = 26
20000149 869da80a 00000018 00000000 0024 USER STARTED 000 useCounter_u
20000148 869da80a 00000019 000009c3 0025 USER STARTED 000 useCounter_u
--> rsh neon akill 24
--> rsh neon akill 25
-->
10.5. Chorus per-actor private data 291
10.5 Chorus per-actor private data
The primitives presented in this section may only be called from a supervisor
actor even though, in the case of padGet and padSet, the target actor may be
a user actor. Any attempt to use one of these primitives from a user actor will
return the PD EUSER error value.

10.5.1 Creating and deleting an actor private key


An actor-speci c data key that will be visible to all actors of the site may be
created by a supervisor actor by a call to the function
#include <pd/chPd.h>
int padKeyCreate(
PdKey *privateKey,
KnPdHdl destructor
);
Once a per-actor key has been created, the value associated to that key is NULL
for every active actor and will be NULL for every actor created later. Actors will
attach a value to a given key by calling the padSet primitive and will retrieve
this value by calling padGet. The values bound to a given key are maintained
on a per-actor basis and will persist for the actor's lifetime.
The optional destructor handler will be called automatically upon an actor's
deletion: if a non-null destructor is associated to a per-actor key and if a non-
null value is associated to that key, the destructor will be called with that value
as argument. Di erent destructors may exist for the same key: in that case,
the order of the calls is unspeci ed.
If the call succeeds, the newly created private key is stored at address pri-
vateKey and the function returns 0. Otherwise a positive error value is returned
(PD ENOKEY, PD ENOMEM, PD ESERVER or PD EUSER).
A supervisor actor can delete an actor's private key by calling the function
#include <pd/chPd.h>
int padKeyDelete(PdKey key);
The various values associated with the deleted key in actors need not be NULL,
but it is the responsibility of actors to free the corresponding space. It is
important to note that the destructor handlers are not called when the key is
deleted (they are only called when an actor terminates). Once a key has been
deleted, no actor should use it: results are unpredictable.
If an error occurs the call returns PD EINVAL, PD ESERVER or PD EUSER according
to the type of the error.
292 Chapter 10. Private data
10.5.2 Setting an actor speci c value
A supervisor actor may associate a value with a given per-actor key by calling
#include <pd/chPd.h>
int padSet(
KnCap *actorCap,
PdKey key,
const void *value
);
The actorCap argument speci es the actor for which the value is set. The value
will typically point to a block of memory previously allocated in the supervisor
address space.
Upon success, the function returns 0. Otherwise, it returns one of the following
positive error values:
return value error type
PD EINVAL invalid key or inconsistent capability
PD ENOMEM not enough memory to associate the value to the key
PD ESERVER the PD manager is not accessible
PD EUSER the call is reserved for supervisor actors
10.5.3 Getting an actor speci c value
A supervisor actor may retrieve the value that is currently associated to a given
per-actor key in an actor by calling the primitive
#include <pd/chPd.h>
void *padGet(
KnCap *actorCap,
PdKey key
);
If the value of key is not the result of a call to the padKeyCreate function or
if it has been deleted by calling padKeyDelete, the e ect is unde ned. If no
value is associated to the key for the given actor, the call returns NULL.
10.5.4 Example
We now develop a small application simulating the distribution of playing cards
for a simpli ed poker game (we do not consider the di erent suits): this appli-
cation uses both per-actor and per-thread private data. An actor will have the
role of a game table where di erent threads will have the role of players.
The actor encapsulates private data that is a deck of 52 cards: when opening
a table, a new shued deck is provided and the main thread just waits for the
game to start:
10.5. Chorus per-actor private data 293
--> cat $CHORUS/sources/privateData/poker/poker.c
#include "utilities.h"
#include <pd/chPd.h>
extern void play(PdKey);
PdKey poker;

void terminate(int *hand) {


/* since we used svPagesAllocate (and not rgnAllocate), we need */
/* to free explicitly the memory at destruction time */
svPagesFree((VmAddr) hand, 0x1000, NULL);
}

void shuffle( ) {
int position, current, number, loop1, loop2, *deck, *privateDeck;
if((privateDeck = padGet(K_MYACTOR, poker)) == NULL) {
printf("allocation of privateHand\n");
svPagesAllocate((VmAddr *) &deck, 53 * sizeof(int),
K_NOWAITFORMEMORY, NULL);
for(loop1 = 0; loop1 < 53; loop1 ++)
deck[loop1] = 0;
deck[53] = 51; /* the bottom of the deck */
padSet(K_MYACTOR, poker, deck);
printf("in actor private should be %p\n", deck);
deck = NULL; /* address of deck is hidden and private now */
privateDeck = padGet(K_MYACTOR, poker);
printf("privateDeck is %p\n", privateDeck);
}
number = 52;
for(loop1 = 1; loop1 <= 13; loop1 ++)
for(loop2 = 0; loop2 < 4; loop2 ++) {
position = rand( ) % number;
current = 0;
while(position != 0) {
if(privateDeck[current] == 0) position --;
current ++;}
while(privateDeck[current] != 0) current ++;
privateDeck[current] = loop1;
number --; }
for(loop1 = 0; loop1 < 52; loop1 ++) {
printf("%2d ", privateDeck[loop1]);
if (loop1 % 20 == 19)
putchar('\n');
}
putchar('\n');
printf("top = %d bottom = %d\n",
privateDeck[52], privateDeck[53]);
}
294 Chapter 10. Private data
main( ) {
KnCap actorCap; actorSelf(&actorCap);
printf("main thread is %d\n", threadSelf( ));
fprintCap(stdout, "there is a deck in actor", &actorCap);
padKeyCreate(&poker, (KnPdHdl) terminate);
srand(actorCap.key.keyTail + actorCap.key.keyHead);
shuffle( );
threadDelay(K_NOTIMEOUT);
/* will resume execution when aborted (see 8.8) */
play(poker);
}

An example of execution is given below:


--> neon-n poker_s.r &
[1] 3340
--> started aid = 25
main thread is 13
there is a deck in actor: 2000014d 869da80a 19 9f3
allocation of privateHand
in actor private should be 0xeb4000
privateDeck is 0xeb4000
6 4 6 11 5 9 10 13 12 1 3 7 9 8 2 11 1 4 12 12
4 3 12 1 6 10 10 13 9 3 3 5 6 2 10 7 2 9 8 4
5 11 2 7 11 1 13 8 13 5 8 7
top = 0 bottom = 51

We now give the code of the play function. Each player corresponds to a thread
executing a player function; it receives a hand of ve cards extracted from the
top of the deck. This hand is per-thread private data: no other thread, except
a cheating one, should access it. The game has been simpli ed as each player
receives ve consecutive cards from the deck. Furthermore, we assume in the
implementation that the underlying system was using a fo scheduling pol-
icy: the synchronization between threads is achieved by yielding the processor
(through threadDelay(K NOBLOCK) calls):
--> cat $CHORUS/sources/privateData/poker/play.c
#include "utilities.h"
#include <pd/chPd.h>
KnThreadLid mainThread;
PdKey handKey; int *deck;
void terminateThread(int *hand) {
free(hand);
printf("thread %d is freeing %p\n", ptdThreadId( ), hand);
if (ptdThreadId( ) == mainThread) {
threadDelay(K_NOBLOCK);
printf("thread %d is exiting\n", ptdThreadId( ));
exit(0); }
10.5. Chorus per-actor private data 295
else {
printf("thread %d is leaving\n", ptdThreadId( ));
threadDelete(K_MYACTOR, K_MYSELF); }

}
void player( ) {
int loop, *hand, *privateHand;
printf("thread %d is entering\n", ptdThreadId( ));
if((privateHand = ptdGet(handKey)) == NULL) {
printf("thread %d is allocating privateHand\n", ptdThreadId( ));
hand = (int *) malloc(5 * sizeof(int));
ptdSet(handKey, hand);
printf("in thread %d privateHand should be %p\n",
ptdThreadId( ), hand);
hand = NULL; /* address of hand is hidden and private now */
printf("in thread %d privateHand is %p\n",
ptdThreadId( ), privateHand = ptdGet(handKey));
}
for(loop = 0; loop < 5; loop ++)
privateHand[loop] = deck[deck[52] ++];
/* the thread is loosing the lead */
threadDelay(K_NOBLOCK);
if(ptdThreadId( ) == mainThread)
printf("----------\n");
printf("thread %d has got: ", ptdThreadId( ));
for(loop = 0; loop < 5; loop ++)
printf("%d ", privateHand[loop]);
putchar('\n");
privateHand = NULL;
threadDelay(K_NOBLOCK);
if(ptdThreadId( ) == mainThread)
printf("----------\n");
ptdThreadDelete( );
}

void play(PdKey deckKey) {


KnCap actorCap;
KnThreadLid thread1, thread2;
mainThread = threadSelf( );
thread1 = newThread((KnPc) player, K_ACTIVE, "thread1");
thread2 = newThread((KnPc) player, K_ACTIVE, "thread2");
printf("threads %d, %d and %d are playing ",
mainThread, thread1, thread2);
deck = padGet(K_MYACTOR, deckKey);
ptdKeyCreate(&handKey, (KnPdHdl) terminateThread);
player( );
}
296 Chapter 10. Private data
In order to resume execution, we just abort the main thread which is blocked in
an abortable call to threadDelay: it then enters the abort state and is awoken:
--> neon threadAbort_u 2000014d 869da80a 19 9f3 13
started aid = 24
--> threads 13, 9 and 14 are playing thread 13 is entering
thread 13 is allocating privateHand
in thread 13 privateHand should be 0x7bbed058
in thread 13 privateHand is 0x7bbed058
thread 9 is entering
thread 9 is allocating privateHand
in thread 9 privateHand should be 0x7bbed088
in thread 9 privateHand is 0x7bbed088
thread 14 is entering
thread 14 is allocating privateHand
in thread 14 privateHand should be 0x7bbed0b8
in thread 14 privateHand is 0x7bbed0b8
----------
thread 13 has got: 6 4 6 11 5
thread 9 has got: 9 10 13 12 1
thread 14 has got: 3 7 9 8 2
----------
thread 13 is freeing 0x7bbed058
thread 9 is freeing 0x7bbed088
thread 9 is leaving
thread 14 is freeing 0x7bbed0b8
thread 14 is leaving
thread 13 is exiting

10.6 Posix private data


10.6.1 Creating a Posix thread-speci c key
A call to the function
#include <pthread.h>
int pthread key create(
pthread key t *key,
void (*destructor)(void *)
);
creates a unique thread data key which will be visible in all threads belonging
to the current actor; this key is stored at the address key. After creation of the
key, the NULL value is attached to the key in all threads of the current actor.
Furthermore, when a new thread is created, the NULL value is attached to all
keys currently de ned in the actor.
10.6. Posix private data 297
The destructor function is automatically invoked when a thread is deleted with
the value associated to the key as the sole argument (if that argument is not
NULL).

10.6.2 Deleting a Posix thread-speci c key


A call to the function
#include <pthread.h>
int pthread key delete(pthread key t key);
deletes the key given as an argument. The key will not be de ned after the call
and any attempt to use it results in unde ned behavior. It is the responsibility
of the application to free all space which could have been allocated to data
associated to the key.
When an actor is deleted, all valid keys in the actor are deleted but no destructor
function is invoked.

10.6.3 Associating a value to a Posix thread-speci c key


A call to the function
#include <pthread.h>
int pthread setspecific(
pthread key t key,
const void *value
);

associates the block of memory pointed to by value to the key (normally obtained
by calling call pthread key create)

10.6.4 Getting the value for a Posix thread-speci c key


A call to the function
#include <pthread.h>
void *pthread getspecific(pthread key t key);
returns a pointer to the bock of memory previously associated to the Key by
a call to int pthread setspecific. The NULL value is returned if no value is
associated to key.
Chapter 11
Local access points (LAPs)
11.1 General overview
Local access points (laps) is a service which provides a software generic interface
for calling handlers (functions) from a user or supervisor actor in a supervisor
one. Therefore, using laps, supervisor actors may provide services to client ac-
tors; in this way, a user actor's thread executing with the user privilege will
gain the supervisor privilege by invoking the supervisor handler associated to
a lap. A lap is a kernel object representing a handler exported by a supervi-
sor actor which can be used by clients for accessing the handler. Laps may
be invoked either explicitly by the speci c lapInvoke primitive or due to an
external event (trap, exception or interrupt - see chapter 14) to which a lap has
been connected. A lap is represented by a lap descriptor which is an opaque
structure generated at lap creation time and whose type is KnLapDesc. This
type of descriptor may be exported to a speci c client or to a nameserver for
more general use.
Additional modules allow binding of local names to the lap (LAPBIND feature)
and a safe on-the- y shutdown (LAPSAFE feature).
We illustrate in the gure given below a typical use of the lap mechanism from
the points of view of server and clients:
 from the point of view of the supervisor actor acting as a server, its main
thread installs the lap. It successively does the following operations:
- lap creation (svLapCreate): it initializes the lap descriptor (lap's entry
point corresponding to the handler, implicit cookie argument which will
be passed as second parameter when the lap will be invoked, and owning
actor);
- lap naming (svLapBind): the lap is registered with the LAPBIND mod-
ule's name server;
299
300 Chapter 11. Local access points (LAPs)
- it kills itself; the actor has no threads, but still exists and encapsulates
the handler code;
 from the client's point of view, that is going to use the service and knows the
lap name:
- the name is resolved by consulting the nameserver (lapResolve);
- the lap is directly invoked (lapInvoke). While executing the handler, the
thread will have the supervisor privilege and the lap's owning actor will
be its execution actor;
- when returning (if the handler returns), the thread will lose the supervisor
privilege (if it belongs to a user actor and had the user privilege as initial
privilege) and resumes execution where it invoked the lap.
lapResolve
(1)
lapInvoke

(3) thread invoking


the lap user actor
user space
supervisor space
supervisor actor
handler address (2) function svLapCreate
cookie (lap handler) svLapBind
name LAPBIND
owning actor module
threadDelete
lap descriptor return main thread
(KnLapDesc) installing the lap
It is important to note that the lap may be invoked concurrently by di erent
threads. Therefore, if it accesses data that can be modi ed by execution, the
corresponding executions should be synchronized.
As a summary, a lap invocation is seen by the client's thread as a remote
procedure call to a di erent actor. From the server point of view the execution
resource (a thread) required to execute the invocation is temporarily provided
by the client for the duration of the invocation.
11.2 General tools
We present here two di erent kinds of tools:
 functions which allow data to be copied between user and supervisor address
spaces. This is a very common operation when a thread belonging to a user
actor is currently executing in a supervisor one through a cross-actor invocation;
 functions allowing manipulation of lap descriptors.
11.2. General tools 301
11.2.1 Copying data between user and supervisor ad-
dress spaces
An important characteristic of a user actor's thread executing with the super-
visor privilege is that it can access the supervisor address space and the user
address space of its home actor at the same time. As said in 7.1.2.1, these
two address spaces may overlap and thus standard bcopy or memcpy functions
cannot be used for this type of copy.
The three memory management modules provide speci c functions for copying
data from user space to supervisor space or the converse safely, that is by re-
turning an error value when an over ow occurs and by correctly interpreting
addresses in the corresponding address space.
a) Copying from user space to supervisor space safely
This type of copy can be requested by a thread executing with the supervisor
privilege by calling the primitive
#include <exec/chExec.h>
int svCopyIn(
VmAddr srcAddr,
void *dstAddr,
int count
);

This call copies count bytes from the address srcAddr in the user address space of
the home actor of the calling thread, into the address dstAddr in the supervisor
address space:
srcAddr User address
space (context
count of the calling actor)

svCopyIn(srcAddr, dstAddr, count)

Supervisor
address space
dstAddr

The call returns K OK if it succeeds, or K EFAULT if a memory access fault is


detected during the copy.
302 Chapter 11. Local access points (LAPs)
A string of characters may also be copied by calling
#include <exec/chExec.h>
int svCopyInString(
VmAddr srcAddr,
void *dstAddr,
int maxCount
);
This call continues the copy until either maxCount bytes have been copied
or a NULL character is encountered. In the rst case, a K ENOMEM error code
is returned, and in the second the number of copied characters is returned.
K EFAULT may also be returned.
b) Copying from supervisor space into user address space safely
A block of data may be copied from the supervisor address space to the home
actor's address space of the calling thread by calling the function
#include <exec/chExec.h>
int svCopyOut(
void *srcAddr,
VmAddr dstAddr,
int count
);
The call returns K OK if it succeeds, or K EFAULT if a memory access fault is
detected during the copy.
11.2.2 Manipulating lap descriptors
Lap descriptors are opaque structures (corresponding to the KnLapDesc opaque
type): di erent functions are provided to manipulate them.
11.2.2.1 Initializing a lap descriptor to zero
A lap descriptor which has previously been allocated may be initialized to 0 by
calling the function
void lapDescZero(KnLapDesc *lapDesc);

11.2.2.2 Testing if a lap descriptor is initialized


A call to the function
int lapDescIsZero(KnLapDesc *lapDesc);
returns a non zero value if the lap descriptor is currently lled with zeroes (it
has been initialized to 0 by a call to lapDescZero or is implemented in a region
initialized to 0). This type of call will return such a value until the lap descriptor
has been successfully initialized by a call to svLapCreate or lapInvoke.
11.3. The server's point of view 303
11.2.2.3 Duplicating a lap descriptor
The contents of a lap descriptor pointed to by oldLapDesc can be duplicated in
a KnLapDesc object pointed to by newLapDesc by a call to
#include <lap/chLap.h>
void lapDescDup(
KnLapDesc *oldLapDesc,
KnLapDesc *newLapDesc
);

11.3 The server's point of view


11.3.1 Creating a local access point
A new lap is created in a supervisor actor by calling the primitive
#include <lap/chLap.h>
int svLapCreate(
KnCap *actorCap,
KnLapHdl lapHdl,
void *cookieDef,
unsigned int options,
KnLapDesc *lapDesc
);
where
 actorCap points to the capability of the actor owning the local access point,
that is, the (supervisor) actor which owns the handler code;
 lapHdl is a pointer to the handler associated to the lap. This handler is a
function whose prototype is
void handler(void *arg, void *cookie);
As we will see, when invoked by a client through a lapInvoke call (see 11.4.2),
the corresponding handler receives as rst argument a value speci ed at call
time and as second argument the cookieDef value speci ed at lap creation.
The semantics bound to the arg parameter received by the handler is left free
to the implementer. It may be an integer or a pointer. Usually it is used as
a pointer to a data structure lled by the invoker and describing in and out
arguments according to a layout de ned by the server at design time;
 the options parameter is either 0 or K LAP SAFE. This ag enforces a strong
checking mechanism during lap invocation. In particular, a request for deleting
the lap by a call to svLapDelete will be synchronized with concurrent lap
304 Chapter 11. Local access points (LAPs)
invocations. This option is mandatory for a lap which is intended to be called
from user actors.
The kern.lap.maxLapSafeNumber con guration variable (whose default value
is 128) de nes the maximum number of laps that may be created with this
option;
 if the call succeeds, the object pointed to by lapDesc receives the lap descriptor
that can be passed to clients' threads for invoking the lap, or can be used for
naming and registering the lap with the LAPBIND nameserver.
If the call succeeds K OK is returned. A negative value points out a failure
(K EINVAL or K NOMEM).
11.3.2 Binding a symbolic name to a lap descriptor
As mentioned, the LAPBIND feature provides a service for naming local access
points and for retrieving the lap descriptor corresponding to a given name.
Without this service, a client wanting to invoke the corresponding handler
should acquire the descriptor directly in some way (for example through a mes-
sage or shared memory). A lap's symbolic name consists of a string of at most
K LAPNAMEMAX characters including a null terminating character (default value
is de ned as 7 in <lap/chLap.h>). It may be bound to a lap by calling the
primitive
#include <lap/chLap.h>
int svLapBind(
KnLapDesc *lapDesc,
char *name,
unsigned int options
);
If the K LAP PROTECTED ag is set in options, the name bound to the lap will
only be visible to trusted threads. Any non trusted thread will receive an error
when trying to resolve a name bound to a lap with this ag using lapResolve.
The call returns K OK if it succeeds. If the name is already in use (bound to
another lap) the call returns the K EBUSY error value, and if the name is too
long, or if there are too many bindings, the K ENOMEM value is returned.
The kern.lap.maxLapBindNumber con guration variable (whose default value
is 256) de nes the maximum number of authorized bindings.
11.3.3 Unbinding a lap's symbolic name
This operation is performed by calling the primitive
int svLapUnbind(char *name);
11.3. The server's point of view 305
The call returns K OK if it succeeds. Otherwise, it returns the negative value
K ENOMEM if the name is too long or, K EINVAL if the name is not bound to any
lap.
11.3.4 Getting handler invoker
When a handler associated to a lap is invoked, it may be interesting in some
circumstances to get the capability of the actor which invoked it, for example
when a unique lap handler is used as the handler associated to di erent events
for di erent client actors. This service is provided by calling the function
#include <exec/chExec.h>
int svGetInvoker(KnCap *actorCap);

If the function is called from outside a lap handler (which must have been
connected by calling svLapCreate, svMsgHandler, svAbortHandler, svExc-
Handler or svTrapConnect) the K EINVAL value is returned.

11.3.5 Deleting a local access point: svLapDelete


A call to the primitive
#include <lap/chLap.h>
int svLapDelete(KnLapDesc *lapDesc);

deletes the local access point whose descriptor is pointed to by lapDesc. If the
lap was created with the K LAP SAFE option, it will be deleted only when all the
current invocations of the lap will have completed.
It is important to note that symbolic names which could have been associated
to the lap by calling svLapBind are not a ected by the lap's deletion and must
be explicitly deleted by calling svLapUnbind.
The call returns K OK if it succeeds, and K EINVAL if lapDesc does not point to
a valid lap descriptor.
11.3.6 Examples
11.3.6.1 Installing a lap in the current actor
We now develop a function which can be used to install a lap in the calling
actor. It will be added to the libutilities.a library and corresponds to the
prototype
int lapInstall(KnLapHdl, void *, char *, char, KnLapDesc *);
which is added in the utilities.h header le associated with our library.
The lapInstall function is passed ve parameters:
306 Chapter 11. Local access points (LAPs)
 the handler address which will be associated to the lap;
 the cookie which will be passed as second argument to the handler upon
invocation;
 a symbolic name which will be bound to the lap;
 if the fourth parameter is 'p' the lap will be protected, therefore name
resolution will only be allowed to trusted threads;
 the fth parameter is a pointer on a KnLapDesc object which should have
been allocated previously.
--> cat $CHORUS/sources/utilities/lib/lapInstall.c
#include "utilities.h"
int lapInstall(KnLapHdl lapHdl, void *cookie, char *lapName,
char mode, KnLapDesc *lapDesc) {
int result, options;
/* the lap is created int safe mode */
result = svLapCreate(K_MYACTOR, lapHdl, (void *)cookie,
K_LAP_SAFE, lapDesc);
if(result < 0) {
fprintf(stderr, "error on svLapCreate: %s\n", strSysError(result));
return result;
}
/* the symbolic name is bound to the lap */
options = (mode == 'p') ? K_LAP_PROTECTED : 0;
result = svLapBind(lapDesc, lapName, options);
if(result < 0) {
fprintf(stderr, "error on svLapBind: %s\n", strSysError(result));
svLapDelete(lapDesc);
return result;
}
return K_OK;
}

11.3.6.2 Copy service between user and supervisor spaces


We now develop a service that clients can invoke through the lap interface and
which allows them rstly to get data from the supervisor address space and
then to move data from their space to the supervisor address space. The same
addr address in the supervisor address space will be used by all threads invoking
the service. Thus, when a thread invokes the service, it will retrieve the data
that the last thread which invoked it wrote at that address in the supervisor
address space. This service uses the svCopyIn and svCopyOut functions that
we presented in 11.2.1.
When a client invokes the service, it sends a couple of addresses packed in a
structure
11.3. The server's point of view 307
/* couple of addresses in actor's user address space */
typedef struct{
char *addr1; /* address in actor's address space holding new value */
char *addr2; /* address in actor's address space for old value */
} twoAddr;

The service will rst copy 16 (0x10) bytes from the addr address in the super-
visor address space into the twoAddrPtr->addr1 address in the user address
space. It will then copy 16 bytes from the twoAddrPtr->addr2 in the user
address space into the addr address in the supervisor address space.
We rst give the code of the function corresponding to the service itself:
--> cat $CHORUS/sources/laps/lapCp/server/userSupCp.c
#include "utilities.h"
typedef struct {
char *addr1;
char *addr2; } twoAddr;
/* a mutex is used for synchronizing executions */
KnMutex mutex = K_KNMUTEX_INITIALIZER;
/* definition and initialization of the supervisor area addr */
char addr[16] = "Initial value";
void handlerCp(twoAddr *twoAddrPtr, void *cookie) {
int result;
fprintf(stderr, "%s\n", cookie);
mutexGet(&mutex);
/* data is copied from addr to the actor's address space */
result = svCopyOut(addr, (VmAddr) twoAddrPtr -> addr1, 16);
if(result < 0) {
fprintf(stderr, "error on svCopyOut: %s\n", strSysError(result));
return; }
/* data is copied from the actor's address space to addr */
result = svCopyIn((VmAddr) twoAddrPtr -> addr2, addr, 16);
if(result < 0) {
fprintf(stderr, "error on svCopyIn: %s\n", strSysError(result));
return; }
mutexRel(&mutex);
}
A lap is associated to this service by executing the lapCp application whose
code is given now:
--> cat $CHORUS/sources/laps/lapCp/server/lapCp.c
#include "utilities.h"
main( ) {
void handlerCp( );
int result;
KnLapDesc lapDesc;
/* request for installing the lap by calling lapInstall */
result = lapInstall((KnLapHdl)handlerCp, "RD-WR", "RdWr", 0, &lapDesc);
308 Chapter 11. Local access points (LAPs)
/* if installation fails, delete region and kill c_actor */
if(result < 0)
exit(2);
fprintf(stderr, "The lap in installed\n");
/* if the lap is installed, the main thread is deleted */
threadDelete(K_MYACTOR, K_MYSELF);
/* actor has no more thread but still exists once the lap is installed */
}

We now execute this application to create the lap:


--> neon-n lapCp_s.r &
[1] 9460
--> started aid = 2
The lap in installed

Once the lap is installed, the main thread of the actor is deleted. The actor no
longer has a thread, but on the other hand it has an address space:
--> rsh neon arun cs -la 2
started aid = 23
ChorusOS r4.0.0 Site 0 Time 45m 21
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000026 869da80a 00000002 00000001 0002 SUP STARTED 000 lapCp_s.r
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
20000026 869da80a def no fac324 0 20000026 869da80a
START SIZE OFFSET ALLOC OPTIONS
7bbe8000 00002000 00000000 00002000 WR SU
7bbed000 00001000 00000000 00001000 WR SU
7bbee000 00009000 00000000 00009000 EX SU
-->

At that point, if we try to create a second actor executing the same command,
we get a failure diagnostic due to a con ict on the lap's name:
--> neon-n lapCp_s.r &
[2] 9464
--> started aid = 23
error on svLapBind: Memory locking conflict

If we delete the c actor owning the lap handler, we can check that the binding
still exists (even if the lap no longer exists):
--> rsh neon akill 2
[1] + Done ........
--> neon-n lapCp_s.r &
[1] 9468
--> started aid = 2
error on svLapBind: Memory locking conflict

An e ective unbinding has to be performed before reusing the symbolic name.


11.3. The server's point of view 309
The svLapUnbind application simply corresponds to a call to the svLapUnbind
primitive and can be used for unbinding a name by creating a c actor for its
execution:
--> cat $CHORUS/sources/utilities/laps/svLapUnbind.c
#include <stdio.h>
#include <chorus.h>
main(int argc, char *argv[ ]) {
int result;
if(argc != 2) {
fprintf(stderr, "Bad usage\n");
exit(2); }
result = svLapUnbind(argv[1]);
if (result != K_OK) {
fprintf(stderr, "error on svLapUnbind: %s\n", strSysError(result));
exit(2); }
printf("Unbinding of %s done\n", argv[1]);
}
--> neon svLapUnbind_s.r RdWr
started aid = 2
Unbinding of RdWr done
--> neon svLapUnbind_s.r RdWr
started aid = 2
error on svLapUnbind: Invalid argument
--> neon-n lapCp_s.r &
[1] 9476
--> started aid = 2
The lap in installed

11.3.6.3 De ning di erent lap handlers in the same actor


This example is intended to illustrate di erent aspects of laps:
 di erent functions of the same supervisor actor may be used as handlers of
di erent laps. Here two handlers are de ned in the same supervisor actor (hdl1
and hdl2);
 the same function may be attached to di erent laps; it is possible to di eren-
tiate the lap which was used for the invocation through the value of the cookie
argument. Here lapin and rabbit are both bound to hdl1: the cookie associ-
ated to the rst one is the "first lap" string and "second lap" is the cookie
of the second. Only one lap handler is associated to hdl2: its symbolic name
is "vuosi" and its cookie is "third lap";
 the service provided by the handlers that are de ned consists simply of dis-
playing general information (invoking actor, local identi er of the thread, ex-
ecution actor's capability, invoked lap) and in stopping the actor identi ed by
K MYACTOR:
310 Chapter 11. Local access points (LAPs)
--> cat $CHORUS/sources/laps/multiLap/server/multiLap.c
#include "utilities.h"
KnLapDesc lap1, lap2, lap3;

void hdl1(char *message, char *cookie) {


KnCap actorCap, invokerActorCap;
KnTimeVal delay;
actorSelf(&actorCap);
fprintf(stderr, "Entering handler1: cookie = %s\n", cookie);
fprintf(stderr, "Current thread: %d\n" , threadSelf( ));
fprintCap(stderr, "in actor (from actorSelf)", &actorCap);
svGetInvoker(&invokerActorCap);
fprintCap(stderr, "Invoker actor", &invokerActorCap);
K_MILLI_TO_TIMEVAL(&delay, 5000);
threadDelay(&delay);
fprintf(stderr, "First argument: %s\n", message);
fprintf(stderr, "Leaving handler1: cookie = %s\n", cookie);
}

void hdl2(char *message, char *cookie) {


KnCap actorCap, invokerActorCap;
int result;
KnTimeVal delay;
actorSelf(&actorCap);
fprintf(stderr, "Entering handler2: cookie = %s\n", cookie);
fprintf(stderr, "Current thread: %d\n", threadSelf( ));
fprintCap(stderr, "in actor (from actorSelf)", &actorCap);
svGetInvoker(&invokerActorCap);
fprintCap(stderr, "Invoker actor", &invokerActorCap);
K_MILLI_TO_TIMEVAL(&delay, 5000);
fprintf(stderr, "First argument: %s\n", message);
result = actorStop(K_MYACTOR);
if(result < 0)
fprintf(stderr, "error on actorStop: %s\n", strSysError(result));
else
fprintf(stderr, "K_MYACTOR has been stopped\n");
fprintf(stderr, "Leaving handler2: cookie = %s\n", cookie);
}

main( ) {
int result;
KnCap actorCap;
actorSelf(&actorCap);
fprintCap(stderr, "Installing lap handlers in", &actorCap);
/* first lap with unprotected binding "lapin" is associated to hdl1 */
result = lapInstall((KnLapHdl) hdl1, "first lap", "lapin", 0, &lap1);
11.3. The server's point of view 311
if (result != K_OK)
exit(1);
/* second lap with protected binding "rabbit" is associated to hdl1 */
result = lapInstall((KnLapHdl) hdl1, "second lap", "rabbit", 'p', &lap2);
if (result != K_OK) {
svLapDelete(&lap1);
svLapUnbind("rabbit");
exit(1); }
/* third lap with unprotected binding "vuosi" is associated to hdl2 */
result = lapInstall((KnLapHdl) hdl2, "third lap", "vuosi", 0, &lap3);
if (result != K_OK) {
svLapDelete(&lap1);
svLapUnbind("rabbit");
svLapDelete(&lap2);
svLapUnbind("cony");
exit(1); }
fprintf(stderr, "*** The three laps are installed ***\n");
threadDelay(K_NOTIMEOUT);
fprintf (stderr, "Lap server is aborting .......\n");
svLapDelete(&lap1);
svLapDelete(&lap2);
svLapDelete(&lap3);
svLapUnbind("lapin");
svLapUnbind("rabbit");
svLapUnbind("vuosi");
}

We now create the c actor which installs and owns the lap handlers:
--> neon-n multiLap_s.r &
[2] 9495
--> started aid = 23
Installing lap handlers in: 20000031 869da80a 17 94
*** The three laps are installed ***

Information about the related actor is displayed. We can check that its main
thread is not deleted. It simply enters an abortable in nite threadDelay, and
when it aborts, it resumes execution and deletes the laps and unbind the sym-
bolic names:
--> rsh neon arun cs -la 23
ChorusOS r4.0.0 Site 0 Time 50m 4
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000031 869da80a 00000017 00000094 0023 SUP STARTED 001 multiLap_s.r
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0008 140 00000000 00000000 ff6330 0- 1- 0 main
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
20000031 869da80a def no fac0ac 0 20000031 869da80a
312 Chapter 11. Local access points (LAPs)
START SIZE OFFSET ALLOC OPTIONS
7bbdb000 00002000 00000000 00002000 WR SU
7bbde000 0000a000 00000000 0000a000 EX SU
7bbec000 00001000 00000000 00001000 WR SU
-->

11.4 The client's point of view


11.4.1 Resolving a name
Before synchronously invoking a lap, a thread needs to acquire its lap descriptor.
If a symbolic name has been bound to the lap, the nameserver may be queried
to get the corresponding lap descriptor. This operation (name resolution) is
performed by calling
#include <lap/chLap.h>
int lapResolve(
KnLapDesc *lapDesc,
char *name,
unsigned int options
);
This call blocks the calling thread until a lap is e ectively associated to the given
name (the call is abortable) except if the K LAP NOBLOCK ag is set in the options
argument. In that case, the call returns immediately and the K EINVAL error
value is returned. If the name is protected (that is, the binding has been done
using the K LAP PROTECTED ag), and the thread is not trusted, the K EPRIV
error value is returned.
11.4.2 Synchronous invocation of a lap handler
Any thread knowing a lap descriptor may invoke the corresponding handler
directly by calling the primitive
#include <lap/chLap.h>
int lapInvoke(
KnLapDesc *lapDesc,
void *argCall
);
In the subsequent handler call whose prototype is
void handler(void *arg, void *cookie);
argCall will be passed as rst parameter and cookieDef as second parameter
(this value is the one speci ed at lap creation).
An e ect of the call is that the actor owning the handler becomes the execution
actor of the calling thread, and the thread executes on its system stack.
11.4. The client's point of view 313
11.4.3 Examples
11.4.3.1 A client of the lapCp service
We give the source of a client of the service; as a rst step, the client resolves
the name and then invokes the lap handler. The client receives a string which
will be written into the supervisor address space as an argument (the length of
this string should be at least 15):
--> cat $CHORUS/sources/laps/lapCp/client/lapCpClient.c
#include "utilities.h"
KnLapDesc lapDesc;
typedef struct{
char *addr1;
char *addr2; } twoAddr;
int result, ind;
twoAddr twoAddrObj;
char response[16];
main(int argc, char *argv[ ]) {
result = lapResolve(&lapDesc, "RdWr", K_LAP_NOBLOCK);
if (result != K_OK) {
fprintf(stderr, "error on lapResolve: %s\n", strSysError(result));
exit(1); }
if(argc != 2 && strlen(argv[1]) < 16) {
fprintf(stderr, "Bad usage\n");
exit(1); }
twoAddrObj.addr1 = response; /* to get the reply */
twoAddrObj.addr2 = argv[1]; /* to send */
result = lapInvoke(&lapDesc, &twoAddrObj);
if (result != K_OK) {
fprintf(stderr, "error on lapInvoke: %s\n", strSysError(result));
exit(1); }
for(ind = 0; ind < 16; ind ++)
fprintf(stderr, "%c", response[ind]);
fprintf(stderr, "\n");
}
-->
Di erent user c actors executing the application are successively created:
--> neon lapCpClient_u TTTTTTTTTTTTTTTTTTTTTTTT
started aid = 22
RD-WR (=printed by the lap handler
Initial value
--> neon lapCpClient_u UUUUUUUUUUUUUUUUUUUUU
started aid = 22
RD-WR (=printed by the lap handler
TTTTTTTTTTTTTTTT (=16 characters written by the previous client
-->
314 Chapter 11. Local access points (LAPs)
Certain speci c situations are worthy of examination:
a) What happens if the lap name is unbound
As we said, what happens depends on the value of the options parameter of
the lapResolve call. As it is written in the code given above (that is, with the
K LAP NOBLOCK ag), the call does not block the calling thread. In the following
trace, we unbind the name and try to execute a client:
--> neon svLapUnbind_s.r RdWr (=
request to unbind RdWr
started aid = 22
Unbinding of RdWr done
--> neon lapCpClient_u VVVVVVVVVVVVVVVVVVVVV
started aid = 22
error on lapResolve: Invalid argument (=
K EINVAL error value
--> rsh neon akill 2 (=
the supervisor actor is killed
-->

The situation is di erent if the lapResolve call does not use this ag. We
produced a lapCpClient2 u code with the call
result = lapResolve(&lapDesc, "RdWr", 0); }
and the execution trace is the following:
--> neon-n lapCp_s.r & (=
a new server is created
[1] 9608
--> started aid = 2
The lap in installed
--> neon svLapUnbind_s.r RdWr (=
unbind the lap
started aid = 22
Unbinding of RdWr done
--> neon lapCpClient2_u VVVVVVVVVVVVVVVVVVVVV
started aid = 22

The calling thread is e ectively blocked. We can observe that killing the actor
owning the lap does not awaken the thread:
-1-> rsh neon akill 2 (=
the server is killed
-1-> rsh neon aps | grep lapCpClient2
0 22 lapCpClient2_u 0 N/A
-1->

We now create (from a di erent shell) a server whose lap is bound to the RdWr
name:
-1-> neon-n lapCp_s.r & (= a new server is created
[1] 12022
-1-> started aid = 2
The lap in installed
RD-WR (=due to the awoken client
The calling thread is awoken and displays the initial string:
11.4. The client's point of view 315
Initial value (=
initial string ( rst client)
--> neon lapCpClient2_u XXXXXXXXXXXXXXXXXXXXXX
started aid = 22
VVVVVVVVVVVVVVVV
RD-WR (=
on the server's screen
b) What happens if the actor owning the handler(s) is deleted?
An interesting situation occurs if the actor owning a lap handler is deleted
but its name is not unbound. The name resolution will succeed and the calling
thread will detect an error when trying to access the handler (as the owning
actor no longer exists).
The following trace illustrates what happens when trying to access the handler
after the last lapCp s.r c actor we created has been killed:
--> rsh neon akill 2
[1] + Done ......... (=
on the server's screen
-1->
--> neon lapCpClient2_u YYYYYYYYYYYYYYYYYYY
started aid = 2
error on lapInvoke: Invalid argument
-->

11.4.3.2 A client of the multiLap service


We develop a multiLapCl application having two arguments:
 the rst is a string which will be passed as argument when invoking the
handler (if the invocation is possible);
 the second is a name bound to a lap handler, in our case it should be lapin1,
rabbit 2 3 or vuosi 4).
--> cat $CHORUS/sources/laps/multiLap/client/multiLapCl.c
#include "utilities.h"
KnCap actorCap;
KnLapDesc lapDesc;
int result;
main(int argc, char *argv[ ]) {
result = lapResolve(&lapDesc, argv[2], K_LAP_NOBLOCK);
if(result < 0) {
fprintf(stderr, "error in lapResolve: %s\n", strSysError(result));
exit(1); }
actorSelf(&actorCap);

1janot lapin invocation is a tribute to Jean de La Fontaine fable "Le Chat, la Belette et
le petit Lapin"
2roger rabbit invocation is a tribute to the Robert Zemeckis hero
3BugsBunny rabbit invocation is a tribute to Tex Avery
4janiksen vuosi invocation is a tribute to the nnish author Arto Paasilinna's book
316 Chapter 11. Local access points (LAPs)
fprintf(stderr, "Resolution of %s ", argv[2]);
fprintCap(stderr, "succeeds in", &actorCap);
fprintf(stderr, "Thread %d is invoking the handler\n", threadSelf( ));
result = lapInvoke(&lapDesc, argv[1]);
if(result < 0) {
fprintf(stderr, "error in lapInvoke: %s\n", strSysError(result));
exit(1); }
actorSelf(&actorCap);
fprintf(stderr, "Thread %d ", threadSelf( ));
fprintCap(stderr, "is back in actor", &actorCap);
}
-->

In the following trace, we invoke the rst lap whose name is rabbit. The
righthand side corresponds to the server's trace and the left to the client's
trace:
--> neon multiLapCl_u janot lapin
started aid = 2
Resolution of lapin succeeds in: 2000003f 869da80a 2 0
Thread 9 is invoking the handler

Entering handler1: cookie = first lap


Current thread: 9
in actor (from actorSelf): 20000031 869da80a 17 94
Invoker actor: 2000003f 869da80a 2 0
First argument: janot
Leaving handler1: cookie = first lap

Thread 9 is back in actor: 2000003f 869da80a 2 0


-->

From these results we notice that:


 it is the thread of the calling actor which executes the handler; no thread has
been created in the supervisor actor;
 the actorSelf call returns the capability of the execution actor;
 the lapInvoke call returns the capability of the calling actor (that is the
calling thread's home actor).
We then run the multiLapCl u command to invoke the second lap handler
named rabbit. We observe that the resolution fails because binding is pro-
tected.
--> neon multiLapCl_u roger rabbit
started aid = 2
error in lapResolve: Privilege violation
We then execute the application in its supervisor version:
11.4. The client's point of view 317
--> neon multiLapCl_s.r roger rabbit
started aid = 2
Resolution of rabbit succeeds in: 20000041 869da80a 2 0
Thread 9 is invoking the handler
Entering handler1: cookie = second lap
Current thread: 9
in actor (from actorSelf): 20000031 869da80a 17 94
Invoker actor: 20000041 869da80a 2 0
First argument: roger
Leaving handler1: cookie = second lap
Thread 9 is back in actor: Thread 9 is back in actor:
-->

Finally, we run the command for invoking the third lap that has been bound to
the vuosi name:
--> neon multiLapCl_u janiksen vuosi
started aid = 2
Resolution of vuosi succeeds in: 20000042 869da80a 2 1
Thread 9 is invoking the handler
Entering handler2: cookie = third lap
Current thread: 9
in actor (from actorSelf): 20000031 869da80a 17 94
Invoker actor: 20000042 869da80a 2 1
First argument: janiksen

We can observe that the execution actor (whose aid is 23) is stopped, not the
home one (whose aid is 2), the K MYACTOR used as argument in the actorStop
call refers to the execution actor. The results of a cs command corroborate this
observation:
-2-> rsh neon arun cs -la 23 (= execution actor (lap server)
started aid = 22
ChorusOS r4.0.0 Site 0 Time 1h 48m 10
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000031 869da80a 00000017 00000094 0023 SUP STOPPED 001 multiLap_s.r
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0008 140 00000000 00000000 ff6330 0- 1- 4 main
......................................
-2-> rsh neon arun cs -la 2 (= home actor (invoker of the lap)
started aid = 22
ChorusOS r4.0.0 Site 0 Time 1h 50m 34
ACTOR-UI KEY LID TYPE STATUS TH# NAME
20000042 869da80a 00000002 00000001 0002 USER STARTED 001 multiLapCl_u
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0009 140 00000000 00000000 ff61a4 0- 1- 0 main
......................................
-2->
318 Chapter 11. Local access points (LAPs)
We create a new c actor invoking the lapin handler:
-2-> neon multiLapCl_u janot lapin
started aid = 22
Resolution of lapin succeeds in: 20000046 869da80a 16 1
Thread 7 is invoking the handler

We can observe that the calling thread is stopped when trying to execute the
handler located in the stopped actor.
We nally restart the home actor by using the actorStart utility presented in
section 6.2.4:
-2-> neon actorStart_u 20000031 869da80a 17 94
started aid = 24
-2->
We observe that the rst thread resumes execution and returns in its home
actor while the second thread enters and executes the handler.
K_MYACTOR has been stopped
Leaving handler2: cookie = third lap
Thread 9 is back in actor: 20000042 869da80a 2 1
Current thread: 7
in actor (from actorSelf): 20000031 869da80a 17 94
Invoker actor: 20000046 869da80a 16 1
First argument: janot
Leaving handler1: cookie = first lap
Thread 7 is back in actor: 20000046 869da80a 16 1

Finally, in the following we run two concurrent executions of the same handler,
and we can observe that executions are nested: this is due to the call to the
threadDelay primitive inside the handler.

-2-> neon-n multiLapCl_s.r roger rabbit& \


neon multiLapCl_s.r BugsBunny rabbit
[1] 9750
started aid = 22
Resolution of rabbit succeeds in: 2000004d 869da80a 16 17b
Thread 7 is invoking the handler
Entering handler1: cookie = second lap
Current thread: 7
in actor (from actorSelf): 20000031 869da80a 17 94
Invoker actor: 2000004d 869da80a 16 17b
started aid = 2
Resolution of rabbit succeeds in: 2000004e 869da80a 2 17c
Thread 9 is invoking the handler
11.4. The client's point of view 319
Entering handler1: cookie = second lap
Current thread: 9
in actor (from actorSelf): 20000031 869da80a 17 94
Invoker actor: 2000004e 869da80a 2 17c
First argument: roger
Leaving handler1: cookie = second lap
Thread 7 is back in actor: 2000004d 869da80a 16 17b
First argument: BugsBunny
Leaving handler1: cookie = second lap
Thread 9 is back in actor: 2000004e 869da80a 2 17c
-2->
Chapter 12
Time management
12.1 Overview
12.1.1 The features
First of all, the executive provides an interrupt level timing service. This low-
level service (which used to correspond to the TIMEOUT feature in previous
releases of the system) provides a traditional one-shot timeout service reserved
to supervisor threads. We will present this service in chapter 14.
In this chapter we will present the services provided by various features which
may be used by threads of any application:
 DATE: maintains the time of day in Universal Time;
 TIMER: provides a high-level timer service allowing user and supervisor threads
to de ne one-shot or periodic timers associated to user-provided handlers. This
service depends on the low-level service provided by the executive;
 VTIMER: allows measurement and timing of thread execution;
 POSIX-TIMERS: provides a compatible implementation of the Posix real-time
clock/timer programming interface.
12.1.2 Describing time intervals
Time intervals are usually described by KnTimeVal objects (within the IPC
feature, they are expressed as integers specifying numbers of milliseconds [see
chapter 13]). We already used this type when delaying threads by calling
threadDelay as de ned as:
struct KnTimeVal flong g
tmSec; long tmNSec; ;

An object of that type corresponds to a duration expressed in seconds ( eld


tmSec) and nanoseconds ( eld tmNSec). Generally, a value of -1 for the tmSec
eld indicates an invalid value.
321
322 Chapter 12. Time management
We have already seen that this type of object can be initialized by calling the
K MILLI TO TIMEVAL macro:
K MILLI TO TIMEVAL(KnTimeVal *tv, int milli);
where milli speci es a number of milliseconds and tv points to the object to
be initialized. Furthermore, the following constants may be used as values for
pointers to KnTimeVal:
- K NOBLOCK: a blocking call won't block the thread, this may be used to poll
an event;
- K NOTIMEOUT: a call will block inde nitely but will be abortable;
- K NOTIMEOUT NOABORT: a call will block inde nitely and will not be abortable.
12.1.3 Time resolution and system time
The functions presented here are exported by the core executive.
The e ective resolution (that is, the smallest di erence between two di erent
values of system time) on a given system may be obtained by calling
#include <exec/chTime.h>
int sysTimeGetRes(KnTimeVal *time);
A call to the primitive
#include <exec/chTime.h>
int sysTime(KnTimeVal *time);
extracts the time elapsed since the system boot (a variable which is initialized
to 0 at boot time is maintained by the kernel) and the value is returned in the
structure pointed to by time. This is the time printed by the cs command.
These two functions returns K OK upon success, or K EFAULT there is data outside
the actor's address space.
The following example illustrates the use of these two general functions:
--> cat $CHORUS/sources/time/time1/time1.c
#include <stdio.h>
#include <stdlib.h>
#include <chorus.h>
main( ) {
KnTimeVal time;
int result;
result = sysTime(&time);
if(result != K_OK) {
fprintf(stderr, "error on sysTime %s\n", strSysError(result));
exit(1); }
printf("system time: %d seconds and %d nanoseconds\n",
time.tmSec, time.tmNSec);
result = sysTimeGetRes(&time);
12.2. The DATE feature 323
if(result != K_OK) {
fprintf(stderr, "error on sysTime %s\n", strSysError(result));
exit(1); }
printf("time resolution: %d seconds et %d nanoseconds\n",
time.tmSec, time.tmNSec);
}
--> neon time1_u
started aid = 23
system time: 12013 seconds and 240000000 nanoseconds
time resolution: 0 seconds et 10000000 nanoseconds
-->

12.2 The DATE feature


As said before, this feature maintains the time of day in Universal Time; this
time is expressed (as in Unix systems) as the number of seconds elapsed since
1 January 1970. We now describe the various functions of the corresponding
API.
12.2.1 Getting universal time resolution: univTimeGetRes
When calling the function,
#include <date/chDate.h>
int univTimeGetRes(KnTimeVal *resolution);

the resolution of the Universal Time is returned in resolution. This value repre-
sents the smallest possible di erence between two distinct values of the universal
time.
12.2.2 Getting time of day: univTime
A call to the function
#include <date/chDate.h>
int univTime(KnTimeVal *currentTime);

returns the time of day in Universal Time in currentTime. If the time of day is
not set yet, currentTime will be set to an invalid value: tmSec = 0 and tmNSec
= K INVALID NSEC.

12.2.3 Setting time of day: ,


univTimeSet univTimeAdjust
The functions described here are reserved for trusted threads.
It is possible to get or set the time-of-day value by calling
324 Chapter 12. Time management
#include <date/chDate.h>
int univTimeSet(
KnTimeVal *oldTime,
KnTimeVal *newTime
);

A call to the function


#include <date/chDate.h>
int univTimeAdjust(
KnTimeVal *oldDelta,
KnTimeVal *newDelta
);

is a request for a progressive adjustment of the time-of-day value.


12.2.4 Example
In the next example we use the ctime function to convert the date to a classical
character string.
--> cat $CHORUS/sources/time/date/date.c
#include <stdio.h>
#include <chorus.h>
#include <date/chDate.h>
int result;
KnTimeVal resol, time1, time2;
main( ) {
result = univTimeGetRes(&resolution);
if (result != K_OK)
printf("error on univTimeGetRes: %s\n", strSysError(result));
else
printf("resolution: %d sec, %d nano\n", resol.tmSec, resol.tmNSec);
result = univTime (&time1);
if (result != K_OK)
printf("error on univTime: %s\n", strSysError(result));
printf("time is: %d seconds, %d nanoseconds\n", time1.tmSec, time1.tmNSec);
printf(" ==> %s\n", ctime(&time1));
time2.tmSec = 40000000; time2.tmNSec = 0;
result = univTimeSet(&time1, &time2);
if (result != K_OK)
printf("error on univTimeSet: %s\n", strSysError(result));
result = univTime (&time1);
if (result != K_OK)
printf("error on univTime: %s\n", strSysError(result));
printf("time is: %d seconds, %d nanoseconds\n", time1.tmSec, time1.tmNSec);
printf(" ==> %s\n", ctime(&time1));
}
12.3. Chorus timers 325
--> neon date_s.r
started aid = 2
resolution: 0 sec, 10000000 nano
time is: 947179797 seconds, 590000000 nanoseconds
==> Thu Jan 6 17:29:57 2000

time is: 40000000 seconds, 10000000 nanoseconds


==> Thu Apr 8 23:06:40 1971
-->

12.3 Chorus timers


12.3.1 Overview
As already stated, the TIMER feature implements a high-level timer service for
user and supervisor actors. It provides the ability to de ne one-shot or periodic
timers. In fact, the timeout related to a timer is noti ed to a thread belonging
to a de ned set of threads. These threads are regular threads which should be
mainly used for executing code at timer expiration.
The central abstraction of this facility consists of timer objects which can be
dynamically created, set and deleted. Timers are attached to an actor and
are locally named in that actor by local identi ers. Di erent timers may be
gathered in a threads pool which establishes a relationship between the actor's
threads and the di erent timers belonging to the pool. These threads should be
coded for processing events corresponding to expirations of timers. The general
organization and use of the mechanisms are summarized in the following gure:

threads in the actor


not belonging to
threads attached to the threadPool
the pool and blocked
timers created in by calling
the pool by calling timerThreadPoolWait
timerCreate and armed
with timerSet
a threadPool object with a KnThreadPool
descriptor in the address space of the actor
actor initialized by calling timerThreadPoolInit
326 Chapter 12. Time management
12.3.2 Initializing a thread pool object
The KnThreadPool type corresponds to a descriptor of a thread pool object
whose structure is opaque to applications. Once allocated in the address space
of the actor, this object has to be initialized before being used by calling the
function
#include <etimer/chEtimer.h>
int timerThreadPoolInit(KnThreadPool *threadPool);
The call returns K OK if successful, or K EFAULT if the address argument is out
of the caller's address space.

12.3.3 Creating a Chorus timer


A timer is created in an actor and inserted into an initialized thread pool object
by calling the function
#include <etimer/chEtimer.h>
int timerCreate(
KnCap *actorCap,
int clockType,
KnThreadPool *threadPool,
void *cookie,
int *timerLi
);

In this call
 actorCap points to the capability of the actor concerned by the timer;
 threadPool points to the descriptor of the thread pool object where the timer
has to be inserted. The corresponding object must have been previously initial-
ized by calling timerThreadPoolInit;
 the only recognized value of clockType is currently K CLOCK REALTIME, which
is the system time returned by sysTime;
 cookie points to a value which will be provided to threads returning from
timerThreadPoolWait calls applied to this threadPool when this timer expires;
 after the call, the value pointed to by timerLi is the local identi er of the
newly created timer.
12.3. Chorus timers 327
The function returns K OK if successful, otherwise a negative value to indicate
the error:
return value error type
K EINVAL clockType is not K CLOCK REALTIME or threadPool
has not been initialized
K ENOMEM no more timers can be created
K EFAULT one address is outside the caller's address space

12.3.4 Deleting a Chorus timer


A timer may be deleted and disarmed if necessary by calling
#include <etimer/chEtimer.h>
int timerDelete(
KnCap *actorCap,
int timerLi
);

The function returns K OK if successful, or a negative value if (K EINVAL, K EFAULT


or K EUNKNOWN) if an error was encountered.

12.3.5 Starting, canceling or querying a timer: timerSet


Once created, a timer has to be initialized. It may be armed in two di erent
ways:
 just for one shot: in this case, when the delay corresponding to the request
expires, the timer is not rearmed;
 for a periodic use: when the delay corresponding to the initial request expires,
the timer is automatically rearmed.
The KnITimer structure that is used for arming a timer has two elds:
- KnTimeVal ITmValue: the initial value of the timer. It de nes the time
at which the rst timeout will occur. This time will be interpreted either
as an absolute time or as a relative time according to a speci c parameter
of the call to the timerSet function used for arming the timer. If the
value of this eld is 0 when arming the timer, it is disarmed;
- KnTimeVal ITmReload : will be used for re-arming the timer. If the cor-
responding time is not 0, the timer will be a periodic timer.
328 Chapter 12. Time management
A timer is armed by calling the primitive
#include <etimer/chEtimer.h>
int timerSet(
KnCap *actorCap,
int timerLi,
int ag,
KnITimer *newDelay,
KnITimer *oldDelay
);
where
 if newDelay value is not NULL, the object it points to de nes the arming value
of the timer identi ed in the actorCap actor by timerLi. If the value is NULL
the timer's current setting is not changed;
 the ag argument may have the value
- K TIMER ABSOLUTE: if newDelay is not NULL, the value of the ITmValue
eld of *newDelay speci es an absolute time (relative to the time main-
tained by sysTime);
- K TIMER INTERVAL (the default value): the value of the ITmValue eld of
*newDelay speci es a time relative to the current time;
 if oldDelay is not NULL and if the timer was already armed, the time remaining
before the next expiration of the timer is returned at the corresponding location
(in any case the returned values are relative values). The function returns K OK
if the call succeeds, or one of the following negative values:
value error type
K EINVAL timerLi is not a valid timer identi cation or
values of newDelay's elds are invalid
K EFAULT one address is outside the caller's address space

12.3.6 Blocking a thread on a thread pool object


A thread may wait for a timer expiration in an initialized thread pool object
by calling the primitive
#include <etimer/chEtimer.h>
int timerThreadPoolWait(
KnThreadPool *threadPool,
void **cookie,
int *overrun,
KnTimeVal *waitLimit
);
12.3. Chorus timers 329
The thread will be blocked until a timer attached to the thread pool expires or
until the delay corresponding to the value pointed to by waitLimit is reached.
When the thread is awoken, the value pointed to by *cookie is equal to the value
pointed to by the cookie argument de ned at timer creation.
It is important to note that only one thread may be active at a time due to
expiration of a timer. This means that if a thread has been awoken because
of a given timer, and has not yet invoked timerThreadPoolWait again, no other
blocked thread will be awoken if the same timer expires again. This type of situ-
ation is called an overrun. Another situation where an overrun occurs is when
a timer expires and no thread is currently blocked in a timerThreadPoolWait.
The system counts the number of overruns encountered. When a thread calls
timerThreadPoolWait and no thread is currently active due to the same timer
and if the counter is not null, this thread is immediately awoken, the value of
the overrun counter is returned at the overrun address, and the kernel's over-
run counter of the timer is reset to 0.
The function normally returns K OK if successful, or a negative value if an error
occurred:
return value error type
K EINVAL threadPool is not an initialized KnThreadPool object
K EFAULT one address is outside the caller's address space

12.3.7 Example
In the example given below:
 if the command is called with an argument, the main thread rst sets the
doThreadDelay variable to 1. Then it initializes a thread pool and creates two
timers which it associates to that thread pool. It creates two new threads which
will execute the timerThread function. The two timers are armed:
- one is periodic: it has an initial relative delay of 3 seconds and a period
of 2 seconds;
- the second one is a one-shot timer whose delay is absolute: this delay is
equal to the current time when it is armed plus 14 seconds.
Finally the main thread is deleted;
 in the timerThread function called by the two new threads, the calling thread
is blocked (without timeout) on timerThreadPoolWait; it waits for expiration
of a delay corresponding to a timer associated to the pool. When awoken, a
thread displays its local identi er, the cookie that is associated to the timer and
the overrun value.
330 Chapter 12. Time management
If the timer was the periodic timer and if doThreadDelay is equal to 1, the
awoken thread sleeps for 7 seconds before calling timerThreadPoolWait once
more. If the timer was the one-shot timer the thread cancels the periodic timer
and exits 20 seconds later.
This can be illustrated as follows:

time
3 5 7 9 11 13 14
--> cat $CHORUS/sources/time/actorTimer/actorTimer.c
#include "utilities.h"
#include <etimer/chEtimer.h>

KnThreadPool pool;
int periodicTimerLocalId, oneshotTimerLocalId;
char *periodic = "Periodic timer"; /* cookie */
char *oneshot = "Oneshot timer"; /* cookie */
int doThreadDelay = 0;

void timerThread( ) {
KnThreadLid myLocalId;
int result;
KnITimer periodicTimer;
char *cookie;
int overrun;
KnTimeVal delay, currentTime;

myLocalId = threadSelf( );
while(1){
/* waiting for a timer expiration */
result = timerThreadPoolWait(&pool, (void **)&cookie, &overrun,
K_NOTIMEOUT);
printf("Thread %d: overrun = %d\n", myLocalId, overrun);
printf(" cookie = %s\n", cookie);
/* the thread has been awoken due to the periodic timer */
if (cookie == periodic) {
/* wait for 7 seconds if doThreadDelay is 1 */
if(doThreadDelay == 1) {
K_MILLI_TO_TIMEVAL(&delay, 7000);
threadDelay(&delay);
}
/* back to the timerThreadPoolWait call */
continue;
}
12.3. Chorus timers 331
/* the thread has been awoken due to the one-shot timer */
else {
periodicTimer.ITmValue.tmSec = 0;
periodicTimer.ITmValue.tmNSec = 0;
periodicTimer.ITmReload.tmSec = 0;
periodicTimer.ITmReload.tmNSec = 0;
timerSet(K_MYACTOR, periodicTimerLocalId, K_TIMER_INTERVAL,
&periodicTimer, NULL);
/* get and print current time */
result = sysTime(&currentTime);
printf(" Current time = %d\n", currentTime.tmSec);
/* wait for 20 seconds before exiting */
K_MILLI_TO_TIMEVAL(&delay, 20000);
threadDelay(&delay);
exit(0);
}
}
}

main(int argc, char *argv[ ]) {


KnThreadLid threadLi1, threadLi2;
KnTimeVal currentTime;
KnITimer periodicTimer, oneshotTimer;
int result;

/* if the command is called with an argument, set doThreadDelay to 1 */


if(argc != 1)
doThreadDelay = 1;
/* initializing the thread pool */
result = timerThreadPoolInit(&pool);
/* creating two timers associated to the thread pool */
result = timerCreate(K_MYACTOR, K_CLOCK_REALTIME, &pool,
periodic, &periodicTimerLocalId);
result = timerCreate(K_MYACTOR, K_CLOCK_REALTIME, &pool,
oneshot, &oneshotTimerLocalId);
/* creating two new threads calling timerThread function */
threadLi1 = newThread((KnPc) timerThread, K_ACTIVE, "thread1");
threadLi2 = newThread((KnPc) timerThread, K_ACTIVE, "thread2");
printf("Threads %d and %d have been created\n",
threadLi1, threadLi2);
/* preparing the periodic timer */
periodicTimer.ITmValue.tmSec = 3;
periodicTimer.ITmValue.tmNSec = 0;
periodicTimer.ITmReload.tmSec = 2;
periodicTimer.ITmReload.tmNSec = 0;
/* arming the relative periodic timer */
timerSet(K_MYACTOR, periodicTimerLocalId, K_TIMER_INTERVAL,
&periodicTimer, NULL);
332 Chapter 12. Time management
/* preparing the one-shot absolute timer */
/* getting current time */
result = sysTime(&currentTime);
printf("Current time in seconds is: %d\n", currentTime.tmSec);
oneshotTimer.ITmValue.tmSec = currentTime.tmSec + 14;
oneshotTimer.ITmValue.tmNSec = currentTime.tmNSec;
oneshotTimer.ITmReload.tmSec = 2;
oneshotTimer.ITmReload.tmNSec = 0;
/* arming the one-shot absolute timer */
timerSet(K_MYACTOR, oneshotTimerLocalId, K_TIMER_ABSOLUTE,
&oneshotTimer, NULL);
/* terminating (by deletion) the main thread */
threadDelete(K_MYACTOR, K_MYSELF);
}

We rst execute the command without argument: when a thread is awoken due
to the periodic timer, it quickly calls timerThreadPoolWait and is blocked on
this call:
--> neon actorTimer_u
started aid = 23
Threads 8 and 9 have been created
Current time in seconds is: 14914
Thread 8: overrun = 0
cookie = Periodic timer
Thread 9: overrun = 0
cookie = Periodic timer
Thread 8: overrun = 0
cookie = Periodic timer
Thread 9: overrun = 0
cookie = Periodic timer
Thread 8: overrun = 0
cookie = Periodic timer
Thread 9: overrun = 0
cookie = Periodic timer
Thread 8: overrun = 0
cookie = Oneshot timer
Current time = 14928
-->
We now execute the command with an argument. When the rst thread is
awoken due to the periodic timer (which occurs at relative time 3), it sleeps
for 7 seconds. At relative times 5, 7 and 9 the periodic timer expires but
no thread can be awoken as the thread which was awoken earlier due to this
timer has not called timerThreadPoolWait. Thus, an overrun with a value of
3 is created. Furthermore, the thread is not blocked when it e ectively calls
timerThreadPoolWait. While it sleeps for 7 seconds, the periodic timer expires
12.4. The VTIMER feature 333
at relative times 11 and 13. The one-shot timer expires at relative time 14 and
the second thread is awoken, it cancels the periodic timer and sleeps for 20
seconds before exiting. When the rst thread calls timerThreadPoolWait (at
approximately relative time 14), it is not blocked and nds an overrun of 2.
Nothing else occurs before the end of the actor:
--> neon actorTimer_u doThreadDelay
started aid = 23
Threads 8 and 7 have been created
Current time in seconds is: 14991
Thread 8: overrun = 0
cookie = Periodic timer
Thread 8: overrun = 3
cookie = Periodic timer
Thread 7: overrun = 0
cookie = Oneshot timer
Current time = 15005
Thread 8: overrun = 2
cookie = Periodic timer
-->

12.4 The VTIMER feature


As we said, this feature exports a number of functions for thread execution
timing, thread accounting and virtual timeouts. We already presented the
threadTimes primitive in 8.10. The various primitives we present here may
be used by supervisor threads.
12.4.1 Virtual timeouts
A virtual timeout corresponds to a speci ed execution time and it may be set by
a supervisor thread to control the execution time either of individual threads or
of entire actors (controlled actors or threads may be user or supervisor threads).
This type of timeout allows the execution time of an individual thread or of a
process including di erent threads to be limited. This limitation may concern
the execution time inside the home actor or may be extended to the whole
execution time, that is, the time consumed when invoking system calls or lap
handlers. A virtual timeout is always relative to the time when it was de ned:
if cpu time has been consumed earlier, it is not counted. A virtual timeout
handler is associated to a virtual timeout that is automatically invoked when
the corresponding time expires.
Virtual timeouts correspond to opaque KnVirtTimeout objects.
The resolution of virtual time values may be obtained by calling the primitive
#include <vtimer/chVtimer.h>
int virtualTimeGetRes(KnTimeVal *resolution);
334 Chapter 12. Time management
The value returned at the address resolution is the smallest possible di erence
(in nanoseconds) between two di erent virtual time values. The function returns
either K OK if successful, or K EFAULT if an error occurs.
12.4.2 Setting a virtual timeout
As we have stated, a virtual timeout can be set to limit the execution of a
given thread or of a whole actor. A general function is provided for setting a
timer either for a complete actor or a speci c thread. Speci c functions are also
provided for de ning each type of virtual timeout.
12.4.2.1 The general interface
A virtual timeout may be de ned by calling the primitive
#include <vtimer/chVtimer.h>
int svVirtualTimeoutSet(
KnCap *actorCap,
KnThreadLid threadLi,
KnVirtTimeout *vTimeout,
KnVirtToHdl handler,
KnTimeVal *cpuTimeLimit,
int ag
);

A virtual timeout is set on the execution time of the thread identi ed by threadLi
in the actor whose capability is pointed to by actorCap. If threadLi has the
K ALLACTORTHREADS value, the timeout will apply to the total execution time of
all threads belonging to the actor. The ag argument may have the following
values:
- K VTIME INTERNAL: only the internal time (the time consumed when executing
in the home actor) is counted in relation to the timeout;
- K VTIME TOTAL: all execution time (internal and external) is counted. It is im-
portant to note that when a thread is executing in an actor other than its home
actor (when executing a system call or a lap handler), the time it consumes is
charged to its home actor and not to the execution actor.
Once the designated thread or threads have consumed *cpuTimeLimit of ad-
ditional execution time (from the time the timer was initialized), the handler
is entered with vTimeout as its sole argument. The handler is executed by the
controlled thread if the timeout is thread-dedicated or by the next executing
one if it is actor-dedicated. This execution is not performed at interrupt level
12.4. The VTIMER feature 335
which means that there is no restriction on the kernel's services that may be
invoked in the handler. If a timeout occurs while the thread is not executing in
its home actor (during cross-actor invocation), the corresponding handler will
be executed when the thread returns to its home actor.
A call will return K OK if successful, or a negative error value:
value error type
K EINVAL no handler has been speci ed or the value of
cpuTimeLimit is not valid or ag is not valid
or threadLi has an incorrect value
K EUNKNOWN actor cannot be reached

12.4.2.2 Setting an actor's virtual timeout


A virtual timer may be set on the total execution time of all threads in a given
actor by calling the primitive
#include <vtimer/chVtimer.h>
int svActorVirtualTimeoutSet(
KnCap *actorCap,
KnVirtTimeout *vTimeout,
KnTimeVal *cpuTimeLimit,
int ag,
KnLapDesc *lapDesc
);
In this call lapDesc designates a lap handler that will be invoked with vTimeout
as a speci c argument when the total amount of CPU time consumed by all the
threads within the actor whose capability is pointed to by actorCap reaches the
cpuTimeLimit value. The lap should have been previously created by calling
svLapCreate. Unlike handlers associated to timeout handlers, the handlers as-
sociated to virtual timeouts are not executed at interrupt level (see 15.4); there
is no restriction concerning the system calls that may be used while executing
the handler. The handler is executed by the next thread of the actor that will
run in its home actor. In fact, virtual timeout handlers are masked during any
cross-actor invocation.
As for svVirtualTimeoutSet, the ag argument may have K VTIME INTERNAL
and K VTIME TOTAL values.
A call will return K OK if successful, or a negative error value:
value error type
K EINVAL lapDesc is not a valid lap descriptor or the value of
cpuTimeLimit is not valid or ag is not valid
K EUNKNOWN actor cannot be reached
336 Chapter 12. Time management
12.4.2.3 Setting a thread's virtual timeout
A virtual timer may be directly set on the total execution time of a given thread
and associated to a given lap handler by calling the primitive
#include <vtimer/chVtimer.h>
int svThreadVirtualTimeoutSet(
KnCap *actorCap,
KnThreadLid threadLi,
KnVirtTimeout *vTimeout,
KnTimeVal *cpuTimeLimit,
int ag,
KnLapDesc *lapDesc
);

Once the thread identi ed by threadLi in the actor whose capability is given by
actorCap has consumed the CPU time speci ed by cpuTimeLimit, the lap han-
dler designated by lapDesc will be called with vTimeout as a speci c argument.
As for the other primitives, ag may be K VTIME INTERNAL or K VTIME TOTAL.
The handler will not be invoked at interrupt level but once the thread executes
in its home actor; thus, during cross-actor invocation, execution of virtual timer
handlers is delayed.
The call normally returns K OK if successful, or a negative value (K EINVAL or
K EUNKNOWN) if an error occurred.

12.4.3 Canceling a virtual timeout


12.4.3.1 The general interface
A timer that has been previously set by calling svVirtualTimeoutSet may be
canceled by calling the primitive
#include <vtimer/chVtimer.h>
int svVirtualTimeoutCancel (KnVirtTimeout *vTimeout);

As a result of the call


 if the virtual timeout is still pending, it is canceled and the call returns the
K OK value;
 if the virtual time has elapsed and the handler has been posted, the call
returns K ETIMEOUT.
12.4. The VTIMER feature 337
12.4.3.2 Cancelling an actor's virtual timeout
An actor-dedicated virtual timer that was previously set by calling svActor-
VirtualTimeoutSet may be canceled by calling the primitive
#include <vtimer/chVtimer.h>
int svActorVirtualTimeoutCancel (KnVirtTimeout *vTimeout);
As a result of the call
 if the virtual timeout is still pending, it is canceled and the call returns the
K OK value;
 if the virtual time has elapsed and the handler has been posted, the call
returns K ETIMEOUT.
12.4.3.3 Canceling a thread's virtual timeout
A thread-dedicated virtual timer that was previously set by calling svThread-
VirtualTimeoutSet may be canceled by calling the primitive
#include <vtimer/chVtimer.h>
int svThreadVirtualTimeoutCancel (KnVirtTimeout *vTimeout);
As a result of the call
 if the virtual timeout is still pending, it is canceled and the call returns the
K OK value;
 if the virtual time has elapsed and the handler has been posted, the call
returns K ETIMEOUT.
12.4.4 Examples
12.4.4.1 The compute function
We used, in di erent examples in chapter 9, a compute function (belonging to
our utilities.a library) which simulates a computation step whose duration
has a random value. Its implementation can be illustrated as follows:

VTIME lap lapInvoke ag set to 1 by timeHdl


set a calling
virtual virtual timeout compute
timer for flag != 0
the client flag == 0
compute
vtimeHdl setTimer
serverCompute supervisor actor actor with a thread calling the compute function
338 Chapter 12. Time management
The compute function invokes a lap named VTIME which sets a virtual timeout
with a value in the range [1..10000] milliseconds. When invoking the lap, the
calling thread transmits its local identi er and the address of a flag variable
which controls a loop; this variable is initialized to 0 and the thread loops while
the variable has this value. The value of the variable will be changed by the
handler associated to this timeout when the virtual timeout expires .
The virtTime.h header le contains the de nition of the type corresponding
to the parameter transmitted at lap invocation
--> cat $CHORUS/sources/time/timeout/include/virtTime.h
#include <stdio.h>
#include <stdlib.h>
#include <chorus.h>
#include <vtimer/chVtimer.h>
typedef struct{
KnThreadLid threadLi;
int *flag;
} timerArg;
-->
The compute function is de ned as:
--> cat $CHORUS/sources/utilities/lib/compute.c
#include "virtTime.h"
int compute(int verbose) {
timerArg arg;
KnLapDesc lapDesc;
int flag = 0, /* will be set to 1 when timeout occurs */
result;
result = lapResolve(&lapDesc, "VTIME", K_LAP_NOBLOCK);
if(result != K_OK)
return -1;
arg.threadLi = threadSelf( );
arg.flag = &flag;
if(lapInvoke(&lapDesc, &arg) != K_OK) return -1;
if(verbose != 0)
fprintf(stderr,
"From compute: thread %d in actor %d starts computation\n",
threadSelf( ), agetId( ));
while(flag == 0) ; /* busy wait for non-zero value of flag */
if(verbose != 0)
fprintf(stderr,
"From compute: thread %d in actor %d ends computation\n",
threadSelf( ), agetId( ));
return 0;
}
-->
12.4. The VTIMER feature 339
A supervisor actor executing the serverCompute application has the following
characteristics:
 the service is provided through a lap named VTIME which is associated to the
setTimer function. This lap is installed by the main thread of a supervisor
actor using the lapInstall function of the libutilities.a library. As it is
currently implemented, the service may be accessed concurrently by 10 di erent
threads. When the lap is invoked, it receives a timerArg argument into which
the local identi er of the calling thread and the address of the flag variable
of that actor are packed. Once the required resources are allocated, the lap
handler installs a virtual timeout where a random value between 1 and 10000
milliseconds is selected;
 the handler associated to this virtual timeout is the vtimeHdl function. In
this function, the value of the flag variable of the target thread is set to 1; this
is done by copying, using svCopyOut, a variable one in the supervisor address
space and whose value is 1 in the flag variable of the target thread.
--> cat $CHORUS/sources/time/timeout/server/serverCompute.c
#include "utilities.h"
#include "virtTime.h"

struct entry{
int *flag;
KnVirtTimeout vTimeout; } vTimeoutEntry[10];
int one = 1;

handler(KnVirtTimeout *vtimeout) {
int ind = 0;
while(&vTimeoutEntry[ind].vTimeout != vtimeout)
ind ++;
svCopyOut(&one, (VmAddr) vTimeoutEntry[ind].flag, sizeof(int));
vTimeoutEntry[ind].flag = NULL;
return;
}

void setTimer(timerArg *arg, char *cookie) {


KnCap actorCap;
KnTimeVal cpuLimit;
int randomDelay, ind, result;
for(ind = 0; ind < 10; ind ++)
if(vTimeoutEntry[ind].flag == NULL) break;
if (ind == 10) {
fprintf(stderr, "too many clients:\n");
svCopyOut(&one, (VmAddr) arg -> flag, sizeof(int));
return; }
result = svGetInvoker(&actorCap);
340 Chapter 12. Time management
if(result < 0) {
fprintf(stderr, "error in svGetInvoker: %s\n", strSysError(result));
svCopyOut(&one, (VmAddr) arg -> flag, sizeof(int));
return;
}
randomDelay = rand( ) % 10000;
K_MILLI_TO_TIMEVAL(&cpuLimit, randomDelay);
vTimeoutEntry[ind].flag = arg -> flag;
result = svVirtualTimeoutSet(&actorCap, arg -> threadLi,
&vTimeoutEntry[ind].vTimeout,
(KnVirtToHdl) handler, &cpuLimit,
K_VTIME_TOTAL);
if(result < 0) {
fprintf(stderr, "error in svVirtualTimeoutSet: %s\n",
strSysError(result));
svCopyOut(&one, (VmAddr) arg -> flag, sizeof(int));
vTimeoutEntry[ind].flag = NULL;
return;
}
return;
}

main( ) {
KnLapDesc lapDesc;
KnCap actorCap;
actorSelf(&actorCap);
srand(actorCap.key.keyHead + actorCap.key.keyTail);
if (lapInstall((KnLapHdl) setTimer, "VTIME", "VTIME", 0, &lapDesc) != K_OK)
exit(1);
threadDelete(K_MYACTOR, K_MYSELF);
}
-->

12.4.4.2 A general service


We develop a service allowing user applications to set virtual timeouts on actors
or individual threads.
12.4.4.3 The server's point of view
The service is accessed through laps. A supervisor actor is created which en-
capsulates a set of virtual timeouts and various lap handlers:
 a pool of virtual timeouts is managed: a timeout is encapsulated in a timeout-
Pool object which contains the virtual timeout, the capability of the actor and
the local identi er of the thread which are target of the virtual timeout. A
free entry in the pool is characterized by the -1 value of the threadLi eld. A
mutex is used when allocating and initializing an object in the pool;
12.4. The VTIMER feature 341
 a number of service functions allow you to get an object in the getTimeout
pool and to retrieve a given pool entry associated to a virtual timeout. These
are derived from the capability of the actor and possibly the local identi er of
a thread (if the corresponding parameter is not -1) that are the target of the
virtual timeout;
 di erent lap descriptors are associated to handlers:
- actorTimerSetLap is a lap bound to the A SET name and is associ-
ated to the actorTimerSetHdl handler. It installs a virtual timeout
for the invoking actor, the duration is passed as an argument expressed
as a number of milli-seconds. The virtual timeout is associated to the
actorTimerExpireLap lap;
- threadTimerSetLap is a lap bound to the T SET name and is associated
to the threadTimerSetHdl handler. It installs a virtual timeout for the
calling thread of the invoking actor, the duration is passed as an argument
expressed as a number of milli-seconds. The virtual timeout is associated
to the threadTimerExpireLap lap;
- actorTimerCancelLap is a lap bound to the A CANC name and is associ-
ated to the actorTimerCancelHdl handler. It cancels the actor-dedicated
virtual timeout used by the invoking actor and returns the object to the
pool;
- threadTimerCancelLap is a lap bound to the T CANC name and is as-
sociated to the threadTimerCancelHdl handler. It cancels the thread-
dedicated virtual timeout used by the calling thread of the invoking actor
and returns the object to the pool;
- actorTimerExpireLap is a lap that has no name and is associated to
the actorTimerExpireHdl handler. It terminates the target actor and
returns the object into the free pool;
- threadTimerExpireLap is a lap that has no name and is associated to
the threadTimerExpireHdl handler. It terminates the target thread and
returns the object into the free pool.
The complete source code of the actor is now given:
--> cat $CHORUS/sources/time/timeHdlActor/server/timeHdlActor.c
#include "utilities.h"
#include <vtimer/chVtimer.h>
typedef struct{
KnVirtTimeout vTimeout;
KnThreadLid threadLi;
KnCap actorCap;
} timeoutPool;
timeoutPool pool[16];
342 Chapter 12. Time management
KnMutex mutex; /* protect allocation of an element in the pool */
KnTimeVal delay;
KnLapDesc threadTimerSetLap, /* for setting a thread timeout */
actorTimerSetLap, /* for setting an actor timeout */
threadTimerCancelLap, /* for canceling a thread timeout */
actorTimerCancelLap, /* for canceling an actor timeout */
threadTimerExpireLap, /* when a thread timeout expires */
actorTimerExpireLap; /* when an actor timeout expires */

/* get a timeout in the pool */


static timeoutPool *getTimeout(KnCap *actorCap) {
int ind;
mutexGet(&mutex);
for(ind = 0; ind < 16; ind ++)
if(pool[ind].threadLi == -1) {
pool[ind].actorCap = *actorCap;
pool[ind].threadLi = threadSelf( );
mutexRel(&mutex);
return (pool + ind); }
mutexRel(&mutex);
return NULL;
}

/* retrieve a timeout set by a thread or an actor */


static timeoutPool *findTimeout(KnCap *actorCap, KnThreadLid threadLi) {
int ind;
for(ind = 0; ind < 16; ind ++)
if(uiEqual(&pool[ind].actorCap.ui, &actorCap -> ui))
if(threadLi == -1 || pool[ind].threadLi == threadLi)
return (pool + ind);
return NULL;
}

/* handler executed when an actor-dedicated virtual timeout expires */


void actorTimerExpireHdl(KnVirtTimeout *vTimeout, char *cookie) {
KnCap invokerCap;
svGetInvoker(&invokerCap);
*((KnThreadLid *)(((int) vTimeout) + sizeof(KnVirtTimeout))) = -1;
fprintCap(stderr, "*** timeout has expired for actor", &invokerCap);
actorDelete(&invokerCap);
}

/* handler executed when a thread-dedicated virtual timeout expires */


void threadTimerExpireHdl(KnVirtTimeout *vTimeout, char *cookie) {
*((KnThreadLid *)(((int) vTimeout) + sizeof(KnVirtTimeout))) = -1;
fprintf(stderr, "*** thread %d is interrupted and deleted\n",
threadSelf( ));
threadDelete(K_MYACTOR, K_MYSELF);
}
12.4. The VTIMER feature 343
/* handler executed to setting an actor-dedicated virtual timeout */
void actorTimerSetHdl(int milliSec, char *cookie) {
timeoutPool *virtTimeout;
KnThreadLid threadLi;
KnCap invokerCap;
svGetInvoker(&invokerCap);
fprintCap(stderr, "*** setting a timer by actor", &invokerCap);
if ((virtTimeout = getTimeout(&invokerCap)) == NULL) {
fprintf(stderr, "No more free timer\n");
actorDelete(&invokerCap);
}
K_MILLI_TO_TIMEVAL(&delay, milliSec);
svActorVirtualTimeoutSet(&invokerCap, &virtTimeout -> vTimeout,
&delay, K_VTIME_INTERNAL, &actorTimerExpireLap);
}

/* handler executed for setting a thread-dedicated virtual timeout */


void threadTimerSetHdl(int milliSec, char *cookie) {
timeoutPool *virtTimeout;
KnCap invokerCap;
svGetInvoker(&invokerCap);
fprintf(stderr, "*** thread %d is setting a timer\n", threadSelf( ));
if ((virtTimeout = getTimeout(&invokerCap)) == NULL) {
fprintf(stderr, "No more free timer\n");
threadDelete(K_MYACTOR, threadSelf( ));
}
K_MILLI_TO_TIMEVAL(&delay, milliSec);
svThreadVirtualTimeoutSet(&invokerCap, threadSelf( ),
&virtTimeout -> vTimeout,
&delay, K_VTIME_INTERNAL, &threadTimerExpireLap);
}

/* handler executed to cancel an actor-dedicated virtual timeout */


void actorTimerCancelHdl(void *arg, char *cookie) {
KnCap invokerCap;
timeoutPool *virtTimeout;
svGetInvoker(&invokerCap);
fprintCap(stderr, "*** canceling a timer by actor", &invokerCap);
if((virtTimeout = findTimeout(&invokerCap, -1)) != NULL) {
svActorVirtualTimeoutCancel(&virtTimeout -> vTimeout);
virtTimeout -> threadLi = -1;
}
}

/* handler executed to cancel a thread-dedicated virtual timeout */


void threadTimerCancelHdl(void *arg, char *cookie) {
KnCap invokerCap;
timeoutPool *virtTimeout;
svGetInvoker(&invokerCap);
344 Chapter 12. Time management
fprintf(stderr, "*** canceling a timer by thread %d ", threadSelf( ));
fprintCap(stderr, "in actor", &invokerCap);
if((virtTimeout = findTimeout(&invokerCap, threadSelf( ))) != NULL) {
svThreadVirtualTimeoutCancel(&virtTimeout -> vTimeout);
virtTimeout -> threadLi = -1;
}
}

/* the main thread: it initializes the variables */


/* and installs the different laps */
main( ) {
int ind;
mutexInit(&mutex);
for(ind = 0; ind < 16; ind ++) pool[ind].threadLi = -1;
svLapCreate(K_MYACTOR, (KnLapHdl) actorTimerSetHdl, "1",
K_LAP_SAFE, &actorTimerSetLap);
svLapUnbind("A_SET");
svLapBind(&actorTimerSetLap, "A_SET", 0);
svLapCreate(K_MYACTOR, (KnLapHdl) threadTimerSetHdl, "2",
K_LAP_SAFE, &threadTimerSetLap);
svLapUnbind("T_SET");
svLapBind(&threadTimerSetLap, "T_SET", 0);
svLapCreate(K_MYACTOR, (KnLapHdl) actorTimerCancelHdl, "3",
K_LAP_SAFE, &actorTimerCancelLap);
svLapUnbind("A_CANC");
svLapBind(&actorTimerCancelLap, "A_CANC", 0);
svLapCreate(K_MYACTOR, (KnLapHdl) threadTimerCancelHdl, "4",
K_LAP_SAFE, &threadTimerCancelLap);
svLapUnbind("T_CANC");
svLapBind(&threadTimerCancelLap, "T_CANC", 0);
svLapCreate(K_MYACTOR, (KnLapHdl) actorTimerExpireHdl, "5",
K_LAP_SAFE, &actorTimerExpireLap);
svLapCreate(K_MYACTOR, (KnLapHdl) threadTimerExpireHdl, "6",
K_LAP_SAFE, &threadTimerExpireLap);
threadDelete(K_MYACTOR, K_MYSELF);
}
-->

We intend to install that service. We rst generate a system where the VTIMER
feature is set (within the standard con gurations it is not):
--> cd $BUILD_DIR
--> configurator -set VTIMER=true
--> make chorus
..............
Image file: /home/jmr/CHORUS/r4/build/chorus.bmon
Finish mkimage
-->
12.4. The VTIMER feature 345
After booting this new con guration on the neon machine, we install the service
we developed:
--> rsh neon arun cs -lM
started aid = 2
ChorusOS r4.0.0 Site 0 Time 48
.. SCHED_FIFO ... MEM_PRM ... LAPSAFE VTIMER ..... LAPSAFE .....
--> neon-n timeHdlActor_s.r &
[1] 11206
--> started aid = 2

12.4.4.4 The client's point of view


A client may access the service through functions de ned in the vTimeoutLib.o
object le which hide the lap interface.
First a header le is provided: it contains the prototypes of the di erent func-
tions:
--> cat $CHORUS/include/vTimeout.h
#include "utilities.h"
int actorTimerSet(int milliSec);
int threadSetTimer(int milliSec);
int actorTimerCancel( );
int threadTimerCancel( );
-->

The di erent functions of the object le whose source is given below are just
stubs that perform the right lap invocations:
--> cat $CHORUS/sources/time/timeHdlActor/lib/vTimeoutLib.c
#include "vTimeout.h"
KnLapDesc lapDesc;
int result;
int actorTimerSet(int milliSec) {
result = lapResolve(&lapDesc, "A_SET", K_LAP_NOBLOCK);
if(result != K_OK)
return result;
result = lapInvoke(&lapDesc, (void *) milliSec);
return result;
}

int threadTimerSet(int milliSec) {


result = lapResolve(&lapDesc, "T_SET", K_LAP_NOBLOCK);
if(result != K_OK)
return result;
result = lapInvoke(&lapDesc, (void *) milliSec);
return result;
}
346 Chapter 12. Time management
int actorTimerCancel( ) {
result = lapResolve(&lapDesc, "A_CANC", K_LAP_NOBLOCK);
if(result != K_OK)
return result;
result = lapInvoke(&lapDesc, NULL);
return result;
}

int threadTimerCancel( ) {
result = lapResolve(&lapDesc, "T_CANC", K_LAP_NOBLOCK);
if(result != K_OK)
return result;
result = lapInvoke(&lapDesc, NULL);
return result;
}

12.4.4.5 Execution
a) The rst client consists of a single thread actor which uses the library func-
tions to interrupt a long loop by setting an actor's dedicated virtual timeout:
--> cat $CHORUS/sources/time/timeHdlActor/client1/monoThreadClient.c
#include "vTimeout.h"
main(int argc, char *argv[ ]) {
int milliSec, result, ind, x;
KnCap actorCap;
actorSelf(&actorCap);
fprintf(stderr, "thread %d ", threadSelf( ));
fprintCap(stderr, "in actor", &actorCap);
milliSec = atoi(argv[1]);
result = actorTimerSet(milliSec);
if(result != K_OK) {
fprintf(stderr, "actorTimerSet: %s\n", strSysError(result));
exit(1);
}
for(ind = 0; ind < 50000000; ind ++) x = 1; /* A LONG LOOP */
fprintf(stderr, "after the loop\n");
result = actorTimerCancel( );
if(result != K_OK) {
fprintf(stderr, "actorTimerCancel: %s\n", strSysError(result));
exit(1);
}
}

In a rst execution, a delay of 1 second is given. This time is long enough to


execute the loop: the virtual timeout is canceled when exiting the loop (the
results from the server are on the left and the results from the client are on the
right):
12.4. The VTIMER feature 347
-1-> neon monoThreadClient_u 1000
started aid = 22
thread 9 in actor: 2000001b 869da80a 16 4b
*** setting a timer by actor: 2000001b 869da80a 16 4b
after the loop
*** canceling a timer by actor: 2000001b 869da80a 16 4b
-->

In the second execution, the delay is now 1/10 second. This delay is not su-
cient and when the virtual timeout expires, the actor is deleted before exiting
its long loop:
-1-> neon monoThreadClient_u 500
started aid = 22
thread 9 in actor: 2000001c 869da80a 16 57
*** setting a timer by actor: 2000001c 869da80a 16 57
*** timeout has expired for actor: 2000001c 869da80a 16 57
-->

b) The second client consists of a multi-threaded actor. The command will


have the following parameters:
- the rst gives the number of annex threads;
- the second de nes a duration expressed as a number of milli-seconds that will
be used for setting one or more virtual timeouts;
- if there is a third parameter, it is a request for setting a virtual timeout for
the whole actor. If this parameter is not used, a virtual timeout is set for the
rst annex thread created.
--> cat $CHORUS/sources/time/timeHdlActor/client2/multiThreadClient.c
#include "vTimeout.h"
int milliSec;
KnThreadLid mainThread, timedThread = -1;

void loop( ) {
int ind;
if(threadSelf( ) == timedThread)
threadTimerSet(milliSec);
for(ind = 0; ind < 500000000; ind ++);
fprintf(stderr, "Thread %d after the loop\n", threadSelf( ));
if(threadSelf( ) == timedThread)
threadTimerCancel( );
threadDelete(K_MYACTOR, K_MYSELF);
}

main(int argc, char *argv[ ]) {


int result, ind, nbThreads;
KnCap actorCap;
KnThreadLid threadLi;
348 Chapter 12. Time management
actorSelf(&actorCap);
fprintf(stderr, "thread %d ", threadSelf( ));
fprintCap(stderr, "in actor", &actorCap);
nbThreads = atoi(argv[1]);
milliSec = atoi(argv[2]);
mainThread = threadSelf( );
if(argc == 3)
actorTimerSet(milliSec);
for(ind = 0; ind < nbThreads; ind ++) {
threadLi = newThread((KnPc) loop, K_ACTIVE, "annex");
fprintf(stderr, "thread %d is created\n", threadLi);
if(ind == 0 && argc != 3)
timedThread = threadLi;
}
loop( );
}
-->
In the rst execution a short delay is given to the rst annex thread which is
interrupted and deleted. Other threads execute normally:
-1-> neon multiThreadClient_u 3 800 1
started aid = 22
thread 9 in actor: 2000001d 869da80a 16 0
thread 8 is created
thread 7 is created
thread 10 is created
Thread 9 after the loop
*** thread 8 is setting a timer (=
rst annex thread
*** thread 8 is interrupted and deleted (=
timer expires
Thread 7 after the loop
Thread 10 after the loop
Once all threads of the actor have been deleted, the client actor still exists and
has no thread, we just kill it:
--> rsh neon arun cs -la 22
started aid = 23
ChorusOS r4.0.0 Site 0 Time 28m 21
ACTOR-UI KEY LID TYPE STATUS TH# NAME
2000001d 869da80a 00000016 00000000 0022 USER STARTED 000 multiThreadClie
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
2000001d 869da80a def no ecee44 0 2000001d 869da80a
START SIZE OFFSET ALLOC OPTIONS
fffe9000 0000c000 00000000 0000c000 WR
ffff5000 0000a000 00000000 0000a000 EX
--> rsh neon akill 22
-->
-1-> (=actor has been killed
12.4. The VTIMER feature 349
In the second execution a short delay is given for the whole actor: the main
thread and the rst annex threads can exit from their loop but the second and
third annex threads are interrupted:
-1-> neon multiThreadClient_u 3 9000
started aid = 22
thread 10 in actor: 20000023 869da80a 16 a4
*** setting a timer by actor: 20000023 869da80a 16 a4
thread 7 is created
thread 8 is created
thread 9 is created
Thread 10 after the loop
Thread 7 after the loop
*** timeout has expired for actor: 20000023 869da80a 16 a4
-1-> (= actor has been deleted
Finally, we have con gured and booted a new system on the neon machine
within the ROUND ROBIN feature:
--> cd $BUILD_DIR
--> configurator -set ROUND_ROBIN=true
--> make chorus
..............
Image file: /home/jmr/CHORUS/r4/build/chorus.bmon
Finish mkimage
-->

Once the system is booted on the neon machine, we start the server:
--> rsh neon arun cs -lM
started aid = 2
ChorusOS r4.0.0 Site 0 Time 1m 8
.... SCHED_CLASS [ FIFO RR RT ] ... MEM_PRM ... VTIMER ....
--> neon timeHdlActor_s.r &
[1] 12010
-->

In that environment we execute a modi ed version of the previous client where


the main thread, once started, adopts the round robin scheduling policy that
the new threads of the actor will inherit:
--> cat $CHORUS/sources/time/timeHdlActor/client3/multiThreadClient_RR.c
........
main(int argc, char *argv[ ]) {
........
KnThreadDefaultSched sched;
sched.tdClass = K_SCHED_RR;
sched.tdPriority = 150;
threadScheduler(K_MYACTOR, K_MYSELF, NULL, &sched);
........
}
350 Chapter 12. Time management
-1-> neon multiThreadClient_RR_u 3 9000
started aid = 22
thread 7 in actor: 20000018 869da80a 16
*** setting a timer by actor: 20000018 869da80a 16 0
thread 9 is created
thread 10 is created
thread 8 is created
*** timeout has expired for actor: 20000018 869da80a 16 0
-1-> (=
actor has been deleted

12.5 Posix interface


12.5.1 Overview
In Posix, the type of clock is identi ed by an object of the type clockid t. The
only clock type supported is the system-wide real-time clock which is de ned
by the constant CLOCK REALTIME.
Periods of time are expressed by objects with the timespec structure de ned
as:
struct timespec {
long ts_sec; /* seconds */
long ts_nsec; /* and nanoseconds */
};

12.5.2 General services


A call to the function
#include <time.h>
int nanosleep(
const struct timespec *rqtp,
struct timespec *rmtp
);

causes the current thread to be suspended from execution for the period speci-
ed by the object pointed to by rqtp. The value speci ed by *rmtp is rounded
to a multiple of the system real-time clock. When returning due to completion
of the wait the call returns 0. If any error occurs, it returns -1 and the errno
variable indicates the error (EFAULT if a pointer is not valid, EINVAL if *rqtp is
invalid or rqtp is NULL, EINTR if the calling thread was aborted). In this last
case, if rmtp is not NULL, the time remaining before the call would have nor-
mally terminated is stored at that location.
12.5. Posix interface 351
A call to the function
#include <time.h>
int clock gettime
clockid t clock id,
struct timespec *tp
);

gets the current value of the clock identi ed by clock id and stores this value at
the address speci ed by tp.

12.5.3 Posix timers


12.5.3.1 Identifying a Posix timer
A timer is identi ed in the system by an object of the type timer t. This type
of identi er will be accurate after its creation (by a call to timer create) until
the deletion of the actor owning it or an explicit deletion of the timer itself (by
a call to timer delete). The only type of timer actually supported is identi ed
by the constant CLOCK REALTIME and corresponds to the real-time clock of the
system.

12.5.3.2 Creating a Posix timer


A Posix timer is created in the current actor by a call to the function
#include <signal.h>
#include <time.h>
int timer create
clockid t clock id, /* CLOCK REALTIME */
struct sigevent *evp,
timer t *timer id
);

When returning from the call, the identi er of the new timer is returned at the
address timer id and the timer is disarmed.
The parameter evp must point to an initialized object allocated by the user
with the sigevent structure de ned as:
352 Chapter 12. Time management
#include <signal.h>
struct sigevent{
int sigev_notify; /* SIGEV_THREAD */
int sigev_signo;
union sigval sigev_value;
void (*sigev_notify_function)(union sigval);
};
union sigval{
int sival_int;
void *sival_ptr;
};

The sigev notify function eld points to the user-de ned function which will
be called asynchronously at expiration of the timer. This call is performed by
a thread which is dynamically created at the creation of the timer.
12.5.3.3 Deleting a Posix timer
This operation is performed by a call to the function
#include <time.h>
int timer delete(timer t timer id);
The e ect of this call is to disarm and delete the timer identi ed by timer id.
The handler thread associated to the timer is deleted (if it is actually running,
its execution will nish before deletion).

12.5.3.4 Arming a Posix timer


The structure itimer is de ned as
struct itimerspec {
struct timespec it_interval;
struct timespec it_value;
};

The function
#include <time.h>
int timer settime(
timer t timer id,
int ags,
const struct itimerspec *new value,
const struct itimerspec *old value
);
12.5. Posix interface 353
allows the timer identi ed by timer id to be armed, reset or disarmed according
to the following:
 if the it value eld of new value is zero, the timer is disarmed;
 if the it value eld of new value is not zero, the timer is armed and the time
of the next expiration is relative to the current time if the TIMER ABSTIME ag
is not set in the ags parameter. Otherwise, the time is an absolute time;
 if the timer is armed and if the it interval eld of new value is not zero, a
periodic timer is speci ed. The value of the eld de nes the period;
 if old value is not NULL, the time remaining before expiration of the previous
timeout will be stored at this address.
The timer gettime function may be used to get the amount of time before
expiration of a given timer:
#include <time.h>
int timer gettime(
timer t timer id,
const struct itimerspec *value
);
Chapter 13
Communications
13.1 Introduction
There are a number of features related to communications services:
 MIPC: allows ecient communication between threads of di erent actors which
make up an application through message spaces (mailboxes (mipc));
 IPC: threads that do not share memory may communicate. It allows asyn-
chronous or synchronous communication and exports the following abstractions:
- messages which are units of communication;
- ports which are point-to-point communication endpoints;
- groups of ports which allow multi-cast communication.
Within the optional IPC REMOTE feature, the services provided by the IPC fea-
ture are available in a distributed, transparent way. Thus, distributed applica-
tions on di erent sites may communicate as if they were located on the same
one;
 within the ACTOR EXTENDED MNGT feature, extended actors may take advan-
tage of the entire C library for dealing with input/output operations compatible
with the Posix standard 1003.1b-1993;
 POSIX MQ: provides an implementation of the real-time message queue inter-
face as de ned by POSIX 1003.1b;
 POSIX SHM provides an implementation of the real-time shared memory inter-
face as de ned by the Posix standard 1003.1b-1993. We already presented this
feature in 7.8;
 POSIX SOCKETS: provides a Posix interface for sockets' system calls (1003.1g
standard). This service is limited to the AF INET domain but may be extended
to the local domain within AF LOCAL.
355
356 Chapter 13. Communications
13.2 Mailboxes
13.2.1 Introduction
With the MIPC feature, an application composed of one or several actors can
create a shared communication environment which allows messages to be ex-
changed between threads in a very ecient way without copying them.
The MIPC feature is based on the concept of message space which encapsulates
within a single entity:
 a set of message pools: these are shared by all the actors of the application.
The various messages exchanged through a message space will be allocated in
one pool. Di erent pools may exist in a given message space (between 1 and
K MSG POOLMAX [16]) and each one is identi ed inside that message space by an
index in the range between 0 and N 1, where N is the number of message
pools in the message space. The main characteristic of a message pool is the
common length of the messages it contains.
A message pool is de ned by two integers which correspond to the elds of the
KnMsgPool structure:
- msgSize de nes the size of each message in the pool;
- msgNumber de nes how many messages belong to the pool;
 a set of message queues: these are used by actors to exchange messages
allocated from the message pools. Messages extracted from message pools and
posted by a thread will be inserted in one queue. A given queue of a message
space may contain messages allocated from di erent pools belonging to that
message space. A message queue is identi ed in a message space by its index
in the range between 0 and M 1, where M is the number of message queues
in the message space.
A message space must rst be created by a thread by calling msgSpaceCreate,
and must be opened by other actors wanting to use it by calling msgSpaceOpen;
these actors are said to share this message space.
A message space is bound to the actor which created it and it is automatically
destroyed when this actor and all actors having opened it are themselves de-
stroyed.
An actor may use (create or open) more than one message queue. However,
there is no way to post a message from a message space (say A) to a queue of
another message space (say B).
13.2.2 Creating a message space
The creation of a message space is performed by a call to the primitive
13.2. Mailboxes 357
#include <mipc/chMipc.h>
int msgSpaceCreate(
KnMsgSpaceId spaceGid,
unsigned int msgQueueNb,
unsigned int msgPoolNb,
const KnMsgPool *msgPools
);
where
 the value spaceGid is chosen by the creator of the message pool space and
constitutes a key which globally identi es the message pool. It will be used in
other actors for opening the message pool and it is currently an integer value.
The value K PRIVATEID corresponds to a private key for the actor (the same
way IPC PRIVATE is used as a private key for System V IPC in UNIX systems);
 msgPoolNb and msgQueueNb give, respectively, the number of message pools
and the number of message queues of the message space to be created;
 msgPools should be an array of msgPoolNb descriptors of message pools.
When returning from a successful call, the message space is created and
 the msgPoolNb pools of messages are created; the message pool whose index
is i contains msgPools[i].msgNumber bu ers. The size of each bu er of that
pool is equal to msgPools[i].msgSize;
 msgQueueNb empty queues are de ned.
The value returned by a successful call is a local identi er which will be used
in the actor to manipulate queues and messages in the pool.
The next diagram shows the contents of the message space after its creation:
A message space after it has been created by the call
msgSpaceCreate( spaceGid, msgQueueNb, msgPoolNb, msgPools)

Message pool of index 0


it contains msgPools[0].msgNumber messages
and all messages have length msgPools[0].msgSize
Message pool of index 1
Set of it contains msgPools[1].msgNumber messages
messages and all messages have length msgPools[1].msgSize
pools
Message pool of index msgPoolNb-1
it contains msgPools[msgPoolNb-1].msgNumber messages
and all messages have length msgPools[msgPoolNb-1].msgSize
msgQueueNb all queues are empty
message queues

Remark: there is no primitive for removing a message space; remember that a


message space is a temporary resource which is removed when all actors using
358 Chapter 13. Communications
it (the actor which created it and actors having opened it) are deleted. There
is also no way to dynamically modify a message space (that is, to change the
number of queues or change a message pool).
The function returns a negative value if an error occurs:
return value interpretation
K EINVAL spaceGid is not K PRIVATEID and is already assigned to a message space
K EOVERLAP insucient space in actor's address space
K EARGS the value of msgQueueNb is 0 or the value
of msgPoolNb not in the range [1:K_MSG_POOLMAX]
K EFAULT one address is out of the address space
K ENOMEM system is unable to allocate memory
Upon creation, the message space is actually allocated and locked in physical
memory. Therefore, it is not possible to create message spaces which do not t
into memory at the time they are created, no further memory allocation will
take place during operations on the message space.
13.2.3 Opening a message space
Once created, a message space may be used by any thread knowing its global
identi er spaceGid (used as rst argument of the msgSpaceCreate primitive)
after it has opened it by means of:
#include <mipc/chMipc.h>
int msgSpaceOpen(KnMsgSpaceId spaceGid);
Upon success, the value returned by the call is a local identi er which will be
used to manipulate queues and messages in the pool. This local identi er may
be used by any thread within the actor. Only one thread of an actor needs to
perform the open operation. Local identi ers are not inherited through actor's
creation (actorCreate, afexec, . . . ).
A negative value is returned when an error occurs:
return value interpretation
K EINVAL spaceGid is not a correct identi er
K EOVERLAP insucient space in address space
K ENOMEM system out of resources

13.2.4 Operations on messages


13.2.4.1 General principles
When a thread of an actor needs to post a message, the sequence of operations
that it performs is the following:
13.2. Mailboxes 359
 it allocates a message from one pool of the message space which ts the
characteristics of the message it wants to post;
 it lls the message with the data it wants to transmit;
 it posts the message to the appropriate queue, and optionally assigns a
priority to it.
Any thread of an actor which has opened the message space may extract a
message from a message queue of that message space. This type of operation is
ecient because there is no copy of the data, the receiving thread is returned
a pointer to the message. It can modify the contents and optionally post the
new message to another queue of the same message space.
Message bu ers have to be explicitly returned to the pools they belong to when
they are no longer used by the receiving threads.
The following gure summarizes this set of operations:
msgFree msgGet (receive)

Not msgRemove
Allocated Queued
allocated
msgAllocate msgPut (send)

13.2.4.2 Allocating a message


The goal here is to get an address in the actor's address space for a message in
the pool of a message space, there is no actual memory allocation (see 13.2.2).
Memory allocation is performed by calling the function
#include <mipc/chMipc.h>
int msgAllocate(
int spaceLid,
unsigned int poolIndex,
unsigned int msgSize,
KnTimeVal *waitLimit,
char **msgAddr
);
where
 spaceLid is the local identi er of the message space which has been returned
by msgSpaceCreate or msgSpaceOpen;
 msgSize is the size of the message the application needs;
 poolIndex is the index of the pool where the message is to be allocated. The
360 Chapter 13. Communications
value given is mandatory (if the pool is empty, the system does not try to
allocate the message to another one) and the msgSize must be consistent with
the size of the message in that pool. If the value K ANY MSGPOOL is used, the
system will allocate the message to the rst pool containing a free message
tting (equal or larger) the requested msgSize size;
 msgAddr is the address of a pointer which will receive the address in the
actor's address space of the allocated message if the allocation is successful. If
the message was previously allocated (and thus, has been freed), its contents
are unde ned (as it contains what was written to it before it was freed).
The call blocks until a message is actually allocated, except if a timeout was
speci ed through the waitLimit argument. The K NOBLOCK value implies an
immediate return with the K ETIMEOUT value instead of blocking the calling if
no message should be immediately available.
A successful call returns K OK and in case of failure a negative error value is
returned:
return value interpretation
K EINVAL spaceLid, poolIndex or waitLimit are invalid
K ESIZE msgSize is not compatible with the request
K ETIMEOUT speci ed timeout expired
K EFAULT msgAddr is not a valid address
K EABORT thread was aborted while waiting for a message
13.2.4.3 Posting a message to a queue
A message that has been allocated by calling msgAllocate or whose address has
been gained through a preceding msgGet, and which may have been initialized
by the application is posted to a given queue of the message space by a call to
the primitive
#include <mipc/chMipc.h>
int msgPut(
int spaceLid,
unsigned int queueIndex,
char *msgAddr,
unsigned int priority
);
If there is at least one thread currently waiting for a message to arrive in the
corresponding message queue (blocked in a msgGet call), one of these threads
is awoken. Otherwise, the message whose address in the actor's address space
is msgAddr, is inserted into the corresponding queue according to its priority
which must be comprised between the values 0 and K MSG PRIOMAX.
13.2. Mailboxes 361
When the message has been successfully posted, the function returns K OK, or a
negative value if the operation failed (K EINVAL, K EOVERLAP, K EARGS, K EFAULT
or K ENOMEM).
13.2.4.4 Getting a message from a queue
When a thread calls the primitive
#include <mipc/chMipc.h>
int msgGet(
int spaceLid,
unsigned int queueIndex,
KnTimeVal *waitLimit,
char **msgAddr,
KnUniqueId *srcActor
);

the rst message, if any, with the highest priority pending behind the message
queue with index queueIndex, in the message space whose local identi er is
spaceLid, is extracted from that message queue. No data is copied but the
address in the address space of the thread's actor is returned through the pointer
msgAddr.
If srcActor is not NULL and the call succeeds, it will contain the source actor's
unique identi er upon return, only the ui eld of its capability is given (the
key eld is secret).
If the message queue is empty, the calling thread is normally blocked except if
a timeout has been speci ed through the waitLimit parameter.
A successful call returns K OK, or one of the following negative values otherwise:
return value interpretation
K EINVAL spaceLid or waitLimit are invalid
K EARGS msgQueueId is not valid
K EFAULT wrong address for msgAddr
K ETIMEOUT timeout expired
K EABORT invoking thread was aborted while waiting

13.2.4.5 Freeing a message


A message that has been previously allocated through msgAllocate or received
through a msgGet call must be explicitly returned to the pool to which it belongs
when all the manipulations on it are nished. This operation is performed by
calling
362 Chapter 13. Communications
#include <mipc/chMipc.h>
int msgFree(
int spaceLid,
char *msgAddr
);

where msgAddr is the address of the message as returned by calls to msgAllo-


cate or msgGet. If any thread is currently waiting for a message from the
pool the message belongs to, the freed message is given to it and the thread is
awoken. Otherwise, the message is appended to the pool.
A successful call returns K OK, or a negative value to indicate error (K EINVAL
or K EBADMSG).
13.2.4.6 Removing a message from a queue
A message may be removed from a message queue by calling the primitive
#include <mipc/chMipc.h>
int msgRemove(
int spaceLid,
unsigned int msgQueueId,
char *msg,
int priority
);

In this type of call


 msg is the address of the message in the actor's address space;
 msgQueueId identi es the message queue in the message space identi ed by
spaceLid;
 priority must be identical to the value of the priority that the message received
when it was posted by calling msgPut.
After the operation, the message is still allocated. If the thread is no longer
using it, it must free it by calling msgFree.
The function returns K OK when successful, or a negative value in case of failure
(K EINVAL if spaceLid is invalid, K EARGS if msgQueueId is not a valid message
space or K EBADMSG if msg is not a valid message address or the message has
already been consumed).
13.2.5 Example
The msgQueue.h header le contains the characteristics of the message space
that will be used:
13.2. Mailboxes 363
--> cat $CHORUS/sources/ipc/msgQueue/include/msgQueue.h
#include "utilities.h"
#include <errno.h>
#include <mipc/chMipc.h>

#define MESSAGE_SPACE 2222 /* message space's global identification */


#define NB_MSG_POOLS 3 /* number of messages pools */
#define NB_MSG_QUEUES 3 /* number of messages queues */
#define TINY_MSG_SZ 1 /* size of tiny messages */
#define SMALL_MSG_SZ 8 /* size of small messages */
#define LARGE_MSG_SZ 256 /* size of large messages */
#define NB_TINY_MSG 1 /* number of tiny messages */
#define NB_SMALL_MSG 4 /* number of small messages */
#define NB_LARGE_MSG 2 /* number of large messages */
#define TINY_POOL 0 /* index of tiny messages pool */
#define SMALL_POOL 1 /* index of small messages pool */
#define LARGE_POOL 2 /* index of large messages pool */
-->

The main thread of a rst actor will create the message space and will post
various messages which will be passed as arguments of the command (a string,
an index of the message queue which will be 1 or 2 and a priority will be given
for every message). A message will be allocated either in the pool of small
messages or in the pool of large messages depending on its size. When it has
posted all its messages, the thread will get a message from the reserved queue
with an index of 0, this message will be posted as a tiny message (extracted
from the corresponding pool) by the rst thread which tries to get a message
into the message space. This ensures that when the main thread terminates, at
least one other thread will be using the message space and thus, the message
space will not be deleted.
--> cat $CHORUS/sources/ipc/msgQueue/msgQueue1/msgQueue1.c
/* the command is called with arguments
message_1 queue_1 priority_1 ... message_n queue_n priority_n */

#include "msgQueue.h"
KnMsgPool msgPool[NB_MSG_POOLS]; /* array of messages pools */
KnCap srcActorCap, actorCap;

main (int argc, char *argv[ ]) {


int ind, result, msgSpaceLi;
unsigned int poolIndex, msgLength, msgPriority, queueIndex;
char *msgAddr;

actorSelf(&actorCap);
fprintCap(stderr, "actor", &actorCap);
364 Chapter 13. Communications
/* the message space is created */
msgPool[LARGE_POOL].msgSize = LARGE_MSG_SZ;
msgPool[LARGE_POOL].msgNumber = NB_LARGE_MSG;
msgPool[SMALL_POOL].msgSize = SMALL_MSG_SZ;
msgPool[SMALL_POOL].msgNumber = NB_SMALL_MSG;
msgPool[TINY_POOL].msgSize = TINY_MSG_SZ;
msgPool[TINY_POOL].msgNumber = NB_TINY_MSG;
msgSpaceLi = msgSpaceCreate(MESSAGE_SPACE, NB_MSG_QUEUES,
NB_MSG_POOLS, msgPool);
if (msgSpaceLi < 0) {
fprintf(stderr, "error on msgSpaceCreate: %s\n",
strSysError(msgSpaceLi));
exit(2);
}
/* every message passed as argument is posted */
for(ind = 1; ind < argc; ind += 3) {
queueIndex = atoi(argv[ind + 1]);
if (queueIndex == 0) {
fprintf(stderr, "queue 0 is reserved\n");
continue;
}
msgPriority = atoi(argv[ind + 2]);
/* a message is allocated in the pool fitting its length */
msgLength = strlen(argv[ind]) + 1; /* + 1 because of null character */
poolIndex = (msgLength <= SMALL_MSG_SZ) ?SMALL_POOL : LARGE_POOL;
result = msgAllocate(msgSpaceLi, poolIndex, msgLength,
K_NOTIMEOUT, &msgAddr);
if(result != K_OK) {
fprintf(stderr, "error on msgAllocate: %s\n", strSysError(result));
exit(2);
}
/* the allocated message is initialized */
strcpy(msgAddr, argv[ind], msgLength);
/* the message is now posted in the right queue */
/* with the given priority */
result = msgPut(msgSpaceLi, queueIndex, msgAddr, msgPriority);
if(result != K_OK) {
fprintf(stderr, "msgPut(%s): %s\n", msgAddr,
strSysError(result));
msgFree(msgSpaceLi, msgAddr);
}
else {
fprintf(stderr, "%s ", (poolIndex == SMALL_POOL)?"small" : "large");
fprintf(stderr,
"message %s has been posted in queue %d with prio %d\n",
msgAddr, queueIndex, msgPriority);
}
}
13.2. Mailboxes 365
/* the thread waits until a client arrives and
posts a message in queue 0 */
result = msgGet(msgSpaceLi, 0, K_NOTIMEOUT, &msgAddr, &srcActorCap.ui);
if (result != K_OK) {
fprintf(stderr, "error on msgGet: %s\n", strSysError(result));
exit(2); }
fprintCap(stderr, "TINY MESSAGE RECEIVED FROM ACTOR", &srcActorCap);
msgFree(msgSpaceLi, msgAddr);
}

In the following execution, we can observe that the main thread (we call it th0)
is blocked after having allocated 4 messages from the pool of small messages
and one from the pool of large messages when it tries to allocate a fth message
in the pool of small messages:
-1-> rsh neon arun cs -lM (=
check if MIPC feature is active
started aid = 23
ChorusOS r4.0.0 Site 0 Time 1h 5m 32
............................... MIPC ............
-1-> neon-n msgQueue1_u aaaaa 1 2 bbbbbbbbbbb 2 3 ccccc 0 2 dddd 2 2 \
eeeee 1 3 ffff 4 1 ggggggg 2 3 hhhhhh 1 1 iiiiiiiiiiiiiiii 2 3&
started aid = 23
actor: 20000038 869da80a 17 1
small message aaaaa has been posted in queue 1 with prio 2
large message bbbbbbbbbbb has been posted in queue 2 with prio 3
queue 0 is reserved
small message dddd has been posted in queue 2 with prio 2
small message eeeee has been posted in queue 1 with prio 3
msgPut(ffff): Arguments inconsistency
small message ggggggg has been posted in queue 2 with prio 3

At that point, the two queues have the following contents that have been ordered
according to the priorities of the messages and the order in which they were
inserted (for each message we have noted its priority and the pool to which it
was allocated):
 queue 1: eeeee [3] (S), aaaaa [2] (S)
 queue 2: bbbbbbbbbbb [3] (L), ggggggg [3] (S), dddd [2] (S)
We now give the code of an application which extracts messages from queues
identi ed by their index that are passed as arguments. The main thread of the
actor rst tries to allocate a message in the pool of tiny messages, this allocation
is done in non blocking mode. Thus, if another thread arrived earlier, there is no
longer a tiny message, the thread is not blocked and can continue. After having
received a message, the message is freed 10 seconds later (this delay simulates
a "long" processing and a side e ect is that the thread yields the processor).
366 Chapter 13. Communications
Thus the thread th0 that posts messages in our example is unblocked and, when
it is scheduled, can post and allocate more messages.
--> cat $CHORUS/sources/ipc/msgQueue/msgQueue2/msgQueue2.c
/* the command is called with arguments queue_1 ... queue_n */
#include "msgQueue.h"
KnCap srcActorCap, actorCap;
int ind, result;
unsigned int queueIndex;
char *msgAddr;
int msgSpaceLi;
KnTimeVal delay;

main (int argc, char *argv[ ]) {


actorSelf(&actorCap);
fprintCap(stderr, "actor", &actorCap);
msgSpaceLi = msgSpaceOpen(MESSAGE_SPACE);
if (msgSpaceLi < 0) {
fprintf(stderr, "error on msgSpaceOpen: %s\n", strSysError(msgSpaceLi));
exit(2);
}
/* a tiny message is allocated and posted in queue 0 */
/* if there is no tiny free message, it means we are not */
/* the first client: allocation is non blocking */
result = msgAllocate(msgSpaceLi, TINY_POOL, 1,
K_NOBLOCK, &msgAddr);
if(result != K_OK)
fprintf(stderr, "someone arrived before: %s\n", strSysError(result));
else {
*msgAddr = 'A';
result = msgPut(msgSpaceLi, 0, msgAddr, 0);
fprintCap(stderr, "TINY HAS BEEN POSTED BY", &actorCap);
if(result != K_OK) {
fprintf(stderr, "msgPut(TINY): %s\n", strSysError(result));
msgFree(msgSpaceLi, msgAddr);
}
}
K_MILLI_TO_TIMEVAL(&delay, 10000);
/* messages are extracted from the specified queues */
for(ind = 1; ind < argc; ind ++) {
queueIndex = atoi(argv[ind]);
if(queueIndex == 0) {
fprintf(stderr, "queue 0 is reserved\n");
continue;
}
result = msgGet(msgSpaceLi, queueIndex, K_NOTIMEOUT,
&msgAddr, &srcActorCap.ui);
13.2. Mailboxes 367
if(result != K_OK)
fprintf(stderr, "msgGet(%d): %s\n", queueIndex,
strSysError(result));
else {
fprintf(stderr,
"message %s from queue %d ", msgAddr, queueIndex);
fprintCap(stderr, "by ", &actorCap);
fprintCap(stderr, "and sent by ", &srcActorCap);
threadDelay(&delay);
fprintf(stderr, "message %s is freed", msgAddr);
msgFree(msgSpaceLi, msgAddr); }
}
}

We now create two c actors for this application whose main threads are des-
ignated by th1 and th2. The information displayed by the threads shows how
messages are extracted from the queues and how a blocked thread is awoken
when messages are freed:
-2-> neon-n msgQueue2_u 1 2 2 & neon-n msgQueue2_u 2 1 2 1 &
[1] 5492
[2] 5493
-2-> started aid = 22
actor: 20000039 869da80a 16 1
TINY HAS BEEN POSTED BY: 20000039 869da80a 16 1
message eeeee from queue 1 by: 20000039 869da80a 16 1
and sent by: 20000038 869da80a 0 0

At that point, th1 has posted the tiny message, has read the rst message in
queue 1 and has called threadDelay; thus, it yields the processor and thread
th2 is activated:
started aid = 24
actor: 2000003a 869da80a 18 1bc
someone arrived before: IPC time-out
message bbbbbbbbbbb from queue 2 by: 2000003a 869da80a 18 1bc
and sent by: 20000038 869da80a 0 0

Thread th2 rst tries to post a tiny message; there are no more messages in
the corresponding pool. As the allocation's request is non blocking, the thread
gets an error return value (K ETIMEOUT) and gets the rst message from queue
2. The thread calls threadDelay. The three threads are sleeping (thread th0
is not awoken since no message has been freed). Thread th1 is awoken rst and
frees the message it extracted from the queue:
message eeeee is freed

Thread th0 is awoken, posts its last two messages, reads the tiny message and
terminates:
368 Chapter 13. Communications
small message hhhhhh has been posted in queue 1 with prio 1
large message iiiiiiiiiiiiiiii has been posted in queue 2 with prio 3
TINY MESSAGE RECEIVED FROM ACTOR: 20000039 869da80a 0 0
-1->

Finally, threads th1 and th2 resume execution:


message ggggggg from queue 2 by: 20000039 869da80a 16 1
and sent by: 20000038 869da80a 0 0
message bbbbbbbbbbb is freed
message aaaaa from queue 1 by: 2000003a 869da80a 18 1bc
and sent by: 20000038 869da80a 0 0
message ggggggg is freed
message iiiiiiiiiiiiiiii from queue 2 by: 20000039 869da80a 16 1
and sent by: 20000038 869da80a 0 0
message aaaaa is freed
message dddd from queue 2 by: 2000003a 869da80a 18 1bc
and sent by: 20000038 869da80a 0 0
message iiiiiiiiiiiiiiii is freed
message dddd is freed
message hhhhhh from queue 1 by: 2000003a 869da80a 18 1bc
and sent by: 20000038 869da80a 0 0
message hhhhhh is freed
[2] - Done ...........
[1] + Done ...........
-2->

13.3 The ChorusOS IPC


13.3.1 The features
13.3.1.1 The IPC feature
This provides synchronous and asynchronous communication services between
actors belonging to the same site and exports the following abstractions:
 ports that are point-to-point communication endpoints;
 port groups that are multi-cast communication endpoints;
 messages that are units of communication.
To build a system allowing communication with the corresponding API, we have
to enable the IPC feature and to build the image:
--> cd $BUILD_DIR
--> configurator -set IPC=true
--> make chorus
...........
-->
13.3. The ChorusOS IPC 369
13.3.1.2 The IPC REMOTE feature
The IPC feature may be extended by IPC REMOTE which allows communication
between actors on di erent sites connected via a communication medium (Eth-
ernet network, ATM or VME bus). Inside a domain, communication between
actors located on di erent systems is provided in a location-transparent way;
this means that applications executing on di erent sites may communicate as
if they were running on the same site (with the same primitives and without
modifying the applications). We now describe how to build a system allowing
transparent communication over an Ethernet network.
a) the rst thing to do is to enable both the IPC and IPC REMOTE features:
--> cd $BUILD_DIR
--> configurator -set IPC=true
--> configurator -set IPC_REMOTE=true
-->

b) the next operation is used to specify whether the communication medium is


provided as an external data link implemented as an independent driver outside
the kernel (Ethernet network for instance) or is internal (VME bus for instance).
This information is provided by setting the value of IPC REMOTE COMM: the value
EXT corresponds to an external data link (VME for a VME bus).
--> configurator -set IPC_REMOTE_COMM=EXT
-->

c) an Ethernet-speci c module must be added to the IOM component, this


will act as the IPC Ethernet data-link driver. To add this module when building
the ChorusOS image, the IOM IPC module is selected:
--> configurator -set IOM_IPC=true
-->

d) the last step before building an image consists of selecting a site number.
Within Chorus IPC every node has a unique site number which is a 32 bits
unsigned integer. The site number is part of the unique identi er of an ob-
ject created on that site (it corresponds to the tail of that unique identi er).
This number is provided to the kernel at boot time and is the value of the
chorusSiteId tunable parameter whose default value is 0. When the target is
booted with the standard Chorus boot monitor, as we did, the site number of
the target is the whole IP address which is retrieved when booting the system
within the RARP protocol. A site number may also be provided by setting the
chorusSiteId tunable parameter as in
--> configurator -set chorusSiteId=3
-->
370 Chapter 13. Communications
It is worth noting that if a static de nition of site numbers is adopted, as many
system images as target machines must be built, each one must de ne a speci c
site number (unlike a dynamic computation of the site number where the same
image may be used for di erent targets). Furthermore, a static value of the
site number takes precedence over the value dynamically provided by the boot
program, if any.
In any case, if no site number is provided (either statically or dynami-
cally, that is, if chorusSiteId is 0) the remote IPC will not be started
even if the feature is con gured.
e) once the system has been booted within the IOM IPC feature and an IP ad-
dress has been assigned to the target, the IOM actor includes a module which
will act as a driver for use by the remote IPC and needs to be attached to an
Ethernet device recognized by the IOM. This operation is performed by using
the ethIpcStackAttach built-in command of C INIT called as
ethIpcStackAttach [ethernet device name]
where ethernet device name is the full pathname of the Ethernet device in the
target device tree which is displayed on the console at boot time. This pathname
needs to be provided only if the target owns several Ethernet controllers.
We summarize below the sequence of operations to boot, on neon and carbon
ix86 targets, a ChorusOS image allowing remote IPC between entities executing
on these systems (we will use the site number provided at boot time and the
same image will be loaded on both systems):
--> cd $BUILD_DIR
--> configurator -set IPC=true
--> configurator -set IPC_REMOTE=true
--> configurator -set IOM_IPC=true
--> make chorus
...............
Start mkimage
...............
Image file: /home/jmr/CHORUS/r4/build/chorus.bmon
Finish mkimage
--> su
Password:
# cd /tftpboot
# cp $BUILD_DIR/chorus.bmon 869DA806.ChorusOS.4.0
# ln 869DA806.ChorusOS.4.0 869DA80a.ChorusOS.4.0
# exit
--> rsh neon reboot
shutdown in progress ..
-->
13.3. The ChorusOS IPC 371
console neon !
My IP 134.157.168.10, RARP Server IP 134.157.168.9
Loading file 869DA80A.ChorusOS.4.0 on server 134.157.168.9: loaded!

............................
C_INIT: rshd started
134.157.168.9:/home/jmr/CHORUS/r4/build/root/ on / (nfs)
C_INIT: init in unsecured mode
--> rsh neon ethIpcStackAttach
ipcEthDtLink Init OK
localIpAddr: 134.157.168.10
bcastIpAddr: 134.157.255.255

IPC stack attached


--> rsh carbon reboot
shutdown in progress ..
-->
[ ]
..... same kind of information .......................
--> rsh carbon ethIpcStackAttach
............................
IPC stack attached
-->

13.3.2 Ports
13.3.2.1 Overview
Ports are resources belonging to an actor which are shared by all threads of the
actor to send and receive messages. At any given time, a port belongs to one
and only one actor. Each actor has a default port created at its creation time.
Furthermore, except for the default port of an actor, a port can migrate (its
owning actor changes).
Messages are always sent from one port to another. A queue of messages is
attached to every port containing all the messages sent to the port and not
yet consumed. This queue can contain at most K CPORTQUEUE messages (it is a
tunable parameter whose default value is 32).
Given the identi cation of a destination port, if the remote communication
feature is enabled, the port localization procedure is the following:
 the kernel looks for the information in a cache where the last localizations
were registered;
 the kernel uses a few hints, such as nding the port on its creation site;
 nally the kernels sends a broadcast localization request.
The use of ports to exchange messages between threads is illustrated by the
372 Chapter 13. Communications
following diagram. Each thread of an actor may use any port belonging to its
owning actor to send messages to another port:

user actor 1: user actor 2:


it owns 2 ports it owns 2 ports
and 2 threads and 1 thread
which can send which can send
from these ports from these ports
and get messages and get messages
from their queues from their queues

supervisor actor:
it owns 1 port
and no thread
at present

13.3.2.2 Identi cation


A port has a unique global identi er of the type KnUniqueId. In addition, it is
locally identi ed by an integer within its owning actor.
The K DEFAULTPORT value identi es the default port of an actor.
Given the local identi cation portLid of a port in the current execution actor
(the port being attached to that actor), a successful call to the function
#include <ipc/chIpc.h>
int portUi(
KnUniqueId *portUid,
KnPortLid portLid
);

writes, at the portUid address, the unique identi er of the port and returns
K OK. In case of failure a negative error code is returned: K EINVAL if portLid
is not a valid identi er within the current actor or K EFAULT if an address is
outside the current actor's address space.
Conversely, a call to
#include <ipc/chIpc.h>
KnPortLid portLi(KnUniqueId *portUid);
returns the local identi er of the port whose unique identi er is pointed to by
portUid (the port must be owned by the current actor). In case of failure a
13.3. The ChorusOS IPC 373
negative error value is returned: K EINVAL if portUid is not the unique identi er
of a port owned by the current actor or K EFAULT.
Here is a simple example using these functions for the default port of an actor:
-2-> cat $CHORUS/sources/ipc/portUi/portUi.c
#include <chorus.h>
KnTimeVal delay;
KnUniqueId portUid;
KnPortLid localId;
main( ) {
portUi(&portUid, K_DEFAULTPORT);
printf("uiHead = %x uiTail = %x\n",
portUid.uiHead, portUid.uiTail);
localId = portLi(&portUid);
printf("K_DEFAULTPORT = %x Local identifier = %x\n",
K_DEFAULTPORT, localId);
K_MILLI_TO_TIMEVAL(&delay, 30000);
threadDelay(&delay); /* sleep for 30 seconds */
}
-2-> neon-n portUi_u &
[1] 5563
-2-> started aid = 24
uiHead = 2000003d uiTail = 869da80a
K_DEFAULTPORT = def0 Local identifier = def0

While the main thread is sleeping, the cs command is used to get information
about the actor:
-1-> rsh neon arun cs -la 11
started aid = 24
ChorusOS r4.0.0 Site 0 Time 3h 3m 20
ACTOR-UI KEY LID TYPE STATUS TH# NAME
2000003e 869da80a 00000018 000001e3 0024 SUP STARTED 001 cs
THREAD-LI PRIORITY TT IT CTX SC-MS-PN
0007 140 00000010 00000010 ff6390 0- 0- 0 main
PORT-UI PORT-LI ENABLED CTX MSG# ACTOR-UI
2000003e 869da80a def no ea6e44 0 2000003e 869da80a
START SIZE OFFSET ALLOC OPTIONS
7bbd1000 00004000 00000000 00004000 WR SU
7bbd5000 00010000 00000000 00010000 EX SU
7bbeb000 00002000 00000000 00002000 WR SU
-1->
-2->

13.3.2.3 Port states


Threads of an actor may receive messages sent to a given port. They may also
receive messages from multiple ports. In order to do so, these ports must be in
374 Chapter 13. Communications
a speci c state called the ENABLED state. Thus, a port may be either the in
ENABLED state or in the DISABLED state.
A call to the function
#include <ipc/chIpc.h>
int portEnable(
KnCap *actorCap,
KnPortLid portLid,
int priority
);

puts the port with local identi er portLid in the actor whose capability is pointed
to by actorCap into the ENABLED state. The priority parameter de nes the
priority of the messages received on the port (a low value indicates a high
priority) with regard to messages received on another "ENABLED" port.
When calling the function
#include <ipc/chIpc.h>
int portDisable(
KnCap *actorCap,
KnPortLid portLid
);

the port that has portLid as its local identi er in the actor whose capability is
pointed to by actorCap enters the DISABLED state.
These two functions return K OK upon success. A negative error code is returned
in case of failure: K EINVAL if portLid is not a valid port within the actor,
K UNKNOWN if actorCap does not correspond to a reachable actor or K EFAULT.

13.3.2.4 Creating a port


A port is created in the actor whose capability is pointed to by actorCap when
calling one of the two functions
#include <ipc/chIpc.h>
int portCreate(
KnCap *actorCap,
KnUniqueId *portUid
);

or
13.3. The ChorusOS IPC 375
#include <ipc/chIpc.h>
int portDeclare(
KnCap *actorCap,
KnUniqueId *portUid
);

With the portCreate function, a unique identi er is allocated by the system


for that port and its value is written at the address given by portUid.
With the portDeclare function, the value pointed to by portUid is given by
the user when calling the function and is used as a unique identi er for the new
port. This identi er should have been built previously by calling the uiBuild
function as described in 5.1.1.6. As using this function is dangerous (due to the
fact that it assumes that the unique identi er is actually unique and correctly
built), calls to portDeclare are restricted to trusted threads (SUPERVISOR
threads or threads belonging to SYSTEM actors).
Once created, the initial state of the port is DISABLED.
If the creation succeeds, the local identi er of the port is returned.
In case of failure, a negative value is returned:

return value interpretation


K EINVAL inconsistent actor capability
K EUNKNOWN unknown actor
K EFAULT wrong address
K ENOMEM system out of resources
K EPRIV unauthorized threads for a call to portDeclare

Here is a piece of code where a port is created in the calling actor:


KnUniqueId portUid; /* port unique identifier */
KnPortLid portLid; /* port local identifier */
portLid = portCreate(K_MYACTOR, &portUid);
if(portLid < 0) {
fprintf(stderr, "error on portCreate: %s\n", strSysError(portLid));
actorDelete(K_MYACTOR);
}

13.3.2.5 Deleting a port


A port is automatically deleted when the actor it belongs to is itself deleted. It
is also possible to ask explicitly for deletion of a port by calling the function
376 Chapter 13. Communications
#include <ipc/chIpc.h>
int portDelete(
KnCap *actorCap,
KnPortLid portLid
);

This call corresponds to a request to delete the port whose local identi er is
portLid in the actor whose capability is pointed to by actorCap.
A side e ect of port deletion is to delete all the messages currently in the
queue associated to that port. Furthermore, all threads blocked in a receive call
(ipcReceive) are awoken and the corresponding calls return the K ENOPORT
value.
Finally, all current RPC transactions (corresponding to calls to ipcCall) are
canceled ed. The K OK value is returned when the call succeeds, or a negative
error value K EINVAL, K EUNKNOWN or K EFAULT otherwise.

13.3.3 Ports groups


13.3.3.1 Introduction
Chorus communication incorporates the concept of groups of ports which allows
messages to be sent to a set of ports. Groups have no real physical existence:
there is no way of enumerating the ports belonging to a group. From a logi-
cal point of view a group is a distributed set of ports identi ed by a capability
encapsulating a unique identi er. This unique identi er is used for sending mes-
sages to a group, and the capability is required to modify the group (inserting
or removing ports in the group).
Chorus groups have the following properties:
 they are open: this means that a message may be sent to a group from a port
which does not belong to that group;
 groups are not necessarily mutually exclusive: a port may belong to several
di erent groups;
 communication with a group may be achieved in di erent modes:
- functional mode: the message will be sent to one port belonging to the
group. This mode may be associative (the port will be chosen according
to its location) or exclusive (the port will be chosen on a site di erent to
the one given). Functional modes may be used for synchronous commu-
nication (RPC mode) or asynchronous communication (simply sending a
message);
13.3. The ChorusOS IPC 377
- broadcast mode: the message will be sent to all ports belonging to the
group. No particular property (like atomicity for example) is guaranteed
by the system. Only asynchronous communication is supported by this
mode.
Broadcast and functional modes are summarized in the following:
- broadcast addressing:
the message is sent to all
ports belonging to the group - functional addressing:
that are accessible the message is sent to one
at present port selected by the kernel
according to a criterion

- an open group: the port from which


the message is sent does not
necessarily belongs to that group
Another important aspect and use of groups is their use for naming services.
When a service is activated in one or several actors, these actors usually create
a port and insert it into a group for which a name (in fact a numeric key similar
to a key for an IPC System V in UNIX systems) has been chosen and is known
by the clients of the services. Thus, a client which wants to consult the service
just has to send a message to the group.
13.3.3.2 Allocating a port group capability
A call to the function
#include <ipc/chIpc.h>
int grpAllocate(
int options,
KnCap *portGroupCap,
int stamp
);
returns the capability of a port group at the address portGroupCap.. Like all
capabilities in the system, this capability is formed by
- a unique identi er for naming the port group when sending messages;
- a key needed for modifying the port group by adding or removing ports.
378 Chapter 13. Communications
The options value allows the type of group to be speci ed:
 K DYNAMIC: it is a dynamic group and each call generates a new group capa-
bility (the stamp argument is ignored in this case);
 K STATUSER: the group is static and is dedicated to user actors. The capabil-
ity returned by the call is not new, but one of a set of preallocated capabilities
for static user group ports. Thus, the system guarantees that each call to that
function with the same value of stamp will return the same capability whatever
the execution site is. The value of stamp is an integer comprised between 0 and
K CGRPSTAMPMAX;
 K STATSYS: the group is static and dedicated to system and supervisor actors.
The capability returned by the call is not new, but one of a set of preallocated
capabilities for static system port groups. Upon success the function returns
K OK, or a negative error value of K EPRIV if the K STATSYS is set in options and
the current thread is not trusted, or K EFAULT.
The following sequence corresponds to a request for allocating a capability for
a static user port group whose stamp is xed:.
KnCap groupCap ; /* port group capability */
int groupLid; /* port group local identifier */
if((groupLid = grpAllocate(K_STATUSER, &groupCap, STAMP)) < 0) {
fprintf(stderr, "error on grpAllocate: %s\n",
strSysError(groupLid));
actorDelete(K_MYACTOR);
}

13.3.3.3 Inserting a port into a port group


A call to the function.
#include <ipc/chIpc.h>
int grpPortInsert(
KnCap *portGroupCap,
KnUniqueId *portUid
);

is a request to insert the port with the unique identi er pointed to by portUid
into the group whose capability is pointed to by portGroupCap.
The function returns K OK when successful, or one of the following negative error
codes:
13.3. The ChorusOS IPC 379
return value interpretation
K EINVAL incorrect group capability
K EUNKNOWN incorrect port unique identi er
K EFAULT address out of address space
K ENOMEM system out of resources
K EPRIV unauthorized threads for system groups
The next sequence follows the two preceding sequences and is a request to insert
the port we created into the group whose capability was just allocated:.
portLid = portCreate(K_MYACTOR, &portUid);
groupLid = grpAllocate(K_STATUSER, &groupCap, STAMP);
if((result = grpPortInsert(&groupCap, &portUid) < 0 )) {
fprintf(stderr, "error on grpPortInsert: %s\n", strSysError(result));
actorDelete(K_MYACTOR); }

13.3.3.4 Removing a port from a port group


This operation is performed by a call to the function.
#include <ipc/chIpc.h>
int grpPortRemove(
KnCap *portGroupCap,
KnUniqueId *portUid
);
The function returns K OK if it succeeded, otherwise it returns one of the fol-
lowing negative values:
return value interpretation
K EINVAL incorrect group capability or port not belonging to the group
K EUNKNOWN incorrect port unique identi er
K EFAULT address out of actor's address space
K EPRIV unauthorized threads for system groups
A typical example where this type of call should be. used is the following:
imagine a replicated service accessible through di erent ports (in di erent actors
and possibly on di erent sites) which are all members of a same group. When
one server receives a request, it may want to avoid receiving new ones for a while
and will thus retire from the group for the duration of the request processing.
13.3.4 Messages
13.3.4.1 Message structure
A message is composed of
 a message body: is a byte string whose length is variable and is limited to
a system value de ned by the K CMSGSIZEMAX symbolic constant (prede ned as
64K bytes). Messages may not contain a body;
380 Chapter 13. Communications
 a message annex: is a byte string that has a xed size prede ned as
K CMSGANNEXSIZE (the ocial prede ned value is 120) if it exists.
It is important to note that messages are not typed.
Messages are described by descriptors whose KnMsgDesc type is de ned as:
typedef structf
unsigned int flags; /* options */
unsigned int bodySize; /* size of message's body */
VmAddr bodyAddr; /* body's address */
VmAddr annexAddr; /* annex address */
KnEvtNum seqNum; /* sequence number */
g KnMsgDesc;
The bodyAddr and annexAddr elds point, respectively, to the starting address
of the message body and the starting address of the message annex in the ac-
tor's address space. A NULL value for one of these elds indicates that the
corresponding part of the message is missing. The bodySize eld gives the
e ective size of the message body (the value 0 is used when the message has no
body).
The flags eld allows the selection of the way in which the body will be trans-
mitted. Its value is built with bitwise operators and the following ags:
 K USERBODY: when the calling thread is a supervisor thread, this ag is used to
indicate that the message body is part of the current user address space. If this
ag is not set, the message body is considered to be part of the kernel address
space. The ag is ignored for threads that are not running with the supervisor
privilege;
 K USERANNEX: this ag is interpreted as K USERBODY but concerns the message
annex;
 K ABORTABLE: when this ag is set, an ipcReceive or ipcCall call is abortable.
The seqNum eld corresponds to the number of messages (including the present
message) that have been received on the port: this eld is updated when the
ipcReceive function returns such a message.

13.3.4.2 Current message


For each thread, there may exist a particular message called current message.
This consists of a system descriptor of the last message received by the thread.
The goal is to simplify common operations like replying to the last message
received.
The current message automatically changes when a new message is received
(when the thread calls ipcReceive), and the current message no longer exists
after replying by invoking ipcReply or after saving the current message by
13.3. The ChorusOS IPC 381
calling ipcSave. It is possible to restore a previously saved message as the
current message by invoking ipcRestore.
The ipcReply, ipcGetData, ipcSysInfo and ipcSave primitives apply only to
the current message: they are presented in 13.3.6.
13.3.5 Destinations and modes
The address destination of a message may be either an individual port or a
port group. The KnIpcDest type is used in the sending primitives (ipcSend or
ipcCall) for specifying the destination of the message. It is de ned as:
typedef struct f
KnUniqueId target; /* port or group unique identi er */
KnUniqueId coTarget; /* for functional modes */
g KnIpcDest;
In the case of the message being sent to a single port, only the target eld is
used: there is no need to ll in the coTarget eld.
When it corresponds to a group identi er, the target eld may be entered
directly using a port group's unique identi er, or it may be derived from this
identi er by calling the function ipcTarget; when calling this function a mode
for the communication with the group is speci ed.
The prototype of the ipcTarget function is de ned as:
#include <ipc/chIpc.h>
int ipcTarget(
KnUniqueId *portGroupUi,
int mode
);
where the mode argument may have the following values:
 K BROADMODE: a broadcast to all members of the group will be performed.
This is the default mode (in case the UI of a group is used without invoking
ipcTarget);
 K FUNCMODE: the message will be sent to one port in the group that is currently
reachable. The member of the group is selected by the system;
 K FUNCUMODE: the message will be sent to one of the reachable ports of the
group that is located on the site speci ed by the coTarget eld. On that site
there is an object (port or actor) whose UI is the one de ned by the coTarget
eld. Alternatively, the coTarget may specify the UI of the site itself. This
addressing mode is called associative functional mode;
 K FUNCXMODE: the message will be sent to one reachable port of the group
whose unique identi er is di erent from the value of coTarget. This mode is
called exclusive functional mode.
382 Chapter 13. Communications
In the following example, which is typical in a client,
 the capability of a group with a given stamp is allocated;
 a message destination is built by applying the port group unique identi er
included in the capability (ui eld of the capability), and an addressing mode
(here K FUNCMODE is selected):
KnCap groupCap; /* port group capability */
int groupLid; /* port group local identifier */
KnIpcDest destination; /* destination of message */

if((groupLid = grpAllocate(K_STATUSER, &groupCap, STAMP)) < 0) {


fprintf(stderr, "error on grpAllocate: %s\n",
strSysError(groupLid));
actorDelete(K_MYACTOR);
}
if((result = ipcTarget(&groupCap.ui, K_FUNCMODE)) < 0) {
fprintf(stderr, "error on ipcTarget: %s\n", strSysError(result));
actorDelete(K_MYACTOR);
}
destination.target = groupCap.ui; /* no coTarget for K_FUNCMODE */

13.3.6 Asynchronous communication


13.3.6.1 Overview
With this type of communication, a message is sent from a port to one port
(in functional mode) or to all ports of a group (in broadcast mode). The cor-
responding service is asynchronous; this means that the sender is not blocked
once the message has been enqueued in its sending list. If the destination is not
located on the same system, no information is given to the sender about the
success of the operation. It is important to note that this mode of communica-
tion does not support a high level of service, messages may be lost or delivered
out of order and it is the responsibility of applications to ensure reliability.
13.3.6.2 Sending a message
A request for sending a message is transmitted to the kernel by calling the
primitive
#include <ipc/chIpc.h>
int ipcSend(
KnMsgDesc *messageDesc,
KnPortLid portLid,
KnIpcDest *destination
);
13.3. The ChorusOS IPC 383
where
 messageDesc points to an object describing the message to be sent: address
and size of the message's body, address and size of the message's annex and
options as described above;
 portLid is the local identi er of a port owned by the sending actor which will
be the source port for the sending operation, if the actor receiving the message
performs an ipcReply operation, the answer will automatically be received on
that port. The K DEFAULTPORT symbolic constant may be used to identify the
actor's default port;
 destination points to an object which describes as we said:
- the destination of the message; this may be a port or a group of ports.
If it is a group, an addressing mode must have been selected by a call to
ipcTarget, otherwise the message will be broadcast;
- a site's identi cation when functional mode has been selected.
When successful, K OK is returned, otherwise a negative value indicates the error:

return value error type


K EFULL local communication system is saturated or destination port is local
and its queue is full
K EINVAL incorrect local port identi er
K EUNKNOWN incorrect group port identi er
K EFAULT address out of address space
K ETOOMUCH body size is too large
We now give the source code of a command for sending a message to a port. The
corresponding command will be passed the unique identi er of the destination
port as rst and second arguments, the annex of the message to send as third
argument and the body of the message as the last argument:
--> cat $CHORUS/sources/ipc/msgSnd/msgSnd.c
/* source of msgSnd command called as */
/* msgSnd uiHead uiTail annexString bodyString */
/* where uiHead and uiTail are hexadecimal values */
/* corresponding to the unique identifier of the destination port */

#include <stdio.h>
#include <chorus.h>
#include <ipc/chIpc.h>
int result;
char annex[K_CMSGANNEXSIZE];
KnCap actorCap;
384 Chapter 13. Communications
KnUniqueId portUid;
KnIpcDest dest;
KnMsgDesc sndMsg;

main(int argc, char *argv[ ]) {


/* the sending thread displays its local identifier */
fprintf(stderr, "Local identifier of thread calling ipcSend: %d\n",
threadSelf( ));
/* decoding the unique identifier of the destination port */
sscanf(argv[1], "%x", &result);
portUid.uiHead = result;
sscanf(argv[2], "%x", &result);
portUid.uiTail = result;
/* preparing the descriptor of the destination of the message */
dest.target = portUid;
/* preparing the descriptor of the message that is to be sent */
/* bzero(annex, 0) */;
strcpy(annex, argv[3]);
sndMsg.annexAddr = (VmAddr) annex;
sndMsg.bodyAddr = (VmAddr) argv[4];
sndMsg.bodySize = strlen(argv[4]) + 1;
sndMsg.flags = 0;
/* the message is sent by calling ipcSend */
result = ipcSend(&sndMsg, K_DEFAULTPORT, &dest);
if(result < 0) {
fprintf(stderr, "error on ipcSend: %s\n", strSysError(result));
exit(2);
}
}

13.3.6.3 Receiving a message


A thread may wait to receive a message on a given port or one port among all
enabled ports. Furthermore, a timeout may be speci ed to de ne a blocking
period. A request to read a message on a port is performed by calling the
function
#include <ipc/chIpc.h>
int ipcReceive(
KnMsgDesc *msgDesc,
KnPortLid *portLid,
int delay
);

 portLid is a pointer to the local identi er of the port on which a message


should be received. If the value pointed to by portLid is K DEFAULTPORT, the
actor's default port is used.
13.3. The ChorusOS IPC 385
If the value pointed to by portLid is K ANYENABLED, the request will be satis ed
as soon as a message is received on one of the actor's enabled ports (see 13.3.1.3)
and in the case where several messages are eligible, the message received on the
port with the highest priority will be selected. The local identi er of the port
where the message was received will be returned at portLid address;
 msgDesc points to a message descriptor used for the message.
When calling the function:
- annexAddr eld should point to a block of K CMSGANNEXSIZE bytes (it has
to be allocated before the call) where the system will copy the message
annex in the actor's address space (if the message contains an annex). If
this eld is NULL the annex, if any, will not be delivered to the actor;
- bodyAddr eld should point to an allocated block with size bodySize
where the message body will be copied by the system. If the allocated size
is smaller than the size of the received message's body only the expected
size is delivered and the remainder is discarded. The message is completely
extracted.
If the bodyAddr eld is NULL and if the message contains a body, the body
is not delivered and is kept in system bu ers, it can be delivered later by
calling ipcGetData (see 13.3.5.3);
 delay is a blocking waiting time: here it is expressed as a number of
milliseconds instead of being expressed through a KnTimeVal structure. A
null value corresponds to a non blocking call and a negative value to an in nite
wait (K NODELAY).
When the call successfully completes,
 if bodyAddr was not NULL, the size of the delivered message body is returned;
this value may be smaller than the true size of the message body if an error
occurred during copying the data;
 if bodyAddr was NULL, the size in bytes of the message body.
The following negative values are returned when an error occurs:
return value error type
K ENOPORT invalid port local identi er
K EINVAL a handler is bound to the port (see 13.4) or K ANYENABLED is used
and no port is enabled
K EFAULT address out of address space
K ETIMEOUT timeout occurred
K EABORT abort during the call
386 Chapter 13. Communications
13.3.6.4 Replying to a message
A call to the function
#include <ipc/chIpc.h>
int ipcReply(KnMsgDesc *msgDesc);
allows a reply to be sent to the thread's current message. The msgDesc argu-
ment points to the reply message that has to be sent: the elds of the object
pointed to by msgDesc have the same interpretation as their related ipcSend.
If the call succeeds the function returns the K OK value, and the thread has no
more current message.
The following negative values are returned when an error occurs:
return value error type
K ENOPORT destination port is local and its queue is full
K EINVAL there is no current message
K EFAULT address out of address space
K ETOOMUCH the message is too big
K EUNKNOWN the destination is unreachable

13.3.6.5 Example
We now develop a server whose architecture corresponds to the following dia-
gram:
portUid : port's global unique identi er
portLid : port's local identi er

Every thread
in the actor request request request
has its own
request
requestBody
reply
on its stack requestBody requestBody requestBody

reply reply reply

annex thread 1 annex thread 2 main thread


replyBody is shared by all threads:
they copy requestBody there
mutex to protect access to replyBody
13.3. The ChorusOS IPC 387
The related service is a simple one which simply intends to illustrate the basic
mechanisms of the IPC and IPC REMOTE features. We will see that when this last
one is active, communication may be performed transparently between threads
executing in actors from di erent systems. This service is a variation on an
echo service and the lap service we developed in 11.3.6.2. Here a client sends
an array of characters and the server replies with the array it received from the
previous client. The server actor is multi-threaded and thus, a mutex is used
to protect the memory area used to retrieve and modify the array shared by all
threads in the actor.
a) The header le
This contains the characteristics of the service:
- the ECHO STAMP constant is the stamp associated to the port group correspond-
ing to the service;
- the BODY SIZE constant is the maximum body size.
--> cat $CHORUS/sources/ipc/service/include/service.h
#include "utilities.h"
#include <sync/chMutex.h>
#define ECHO_STAMP 2345 /* service port group's stamp */
#define BODY_SIZE 256 /* maximum body size */
-->

b) The server
An echo server creates a new port that it inserts in the static user port group
corresponding to the ECHO STAMP value selected. Then it enters an in nite loop
where it waits for messages. Each time a message arrives, it echoes it on its
standard error output and sends it back to the client:
--> cat $CHORUS/sources/ipc/service/server/server.c
#include "service.h"
#define N_THREADS 2 /* number of annex threads */
KnUniqueId portUid; /* port's unique identifier */
KnPortLid portLid; /* port's local identifier */
KnCap groupCap ; /* port group capability */
int groupLid; /* port group local identifier */
KnMutex mutex = K_KNMUTEX_INITIALIZER; /* to synchronize executions */
char replyBody[BODY_SIZE] = "Initial value"; /* shared by all threads */

void echoService( ) {
int result; /* return values */
char requestBody[BODY_SIZE];
KnMsgDesc request, /* received message's descriptor */
reply; /* reply message's descriptor */
388 Chapter 13. Communications
/* Prepare the message descriptor for ipcReceive */
request.bodyAddr = (VmAddr) requestBody;
request.bodySize = BODY_SIZE; /* size of allocated space for message */
request.annexAddr = NULL; /* no annex in the message */
/* Prepare the message descriptor for ipcReply */
reply.bodyAddr = (VmAddr) replyBody;
reply.bodySize = BODY_SIZE; /* size of allocated space for message */
reply.annexAddr = NULL; /* no annex in the message */
/* enter the service loop */

while(1) {
bzero(requestBody, BODY_SIZE); /* the buffer is filled with 0 */
/* read a message from the port */
result = ipcReceive(&request, &portLid, -1);
if(result < 0) {
fprintf(stderr, "++ thread %d ++: error on ipcReceive: %s\n",
threadSelf( ), strSysError(result));
continue;
}
fprintf(stderr, "++ thread %d has received a message\n", threadSelf( ));
fprintf(stderr, " size = %d\n", result);
fprintf(stderr, " body: %s\n", request.bodyAddr);
mutexGet(&mutex);
/* the message is sent back as a reply */
if((result = ipcReply(&reply)) < 0) {
fprintf(stderr, "error on ipcReply: %s\n", strSysError(result));
continue;
}
bcopy(requestBody, replyBody, BODY_SIZE);
mutexRel(&mutex);
}
}

main( ) {
int ind, result;
/* create a new port in the actor */
if((portLid = portCreate(K_MYACTOR, &portUid)) < 0 ) {
fprintf(stderr, "error on portCreate: %s\n",
strSysError(portLid));
actorDelete(K_MYACTOR);
}
/* Allocate a port group's capability */
if((groupLid = grpAllocate(K_STATUSER, &groupCap, ECHO_STAMP)) < 0) {
fprintf(stderr, "error on grpAllocate: %s\n",
strSysError(groupLid));
actorDelete(K_MYACTOR);
}
13.3. The ChorusOS IPC 389
/* insert the port in the group */
if((result = grpPortInsert(&groupCap, &portUid) < 0 )) {
fprintf(stderr, "error on grpPortInsert: %s\n", strSysError(result));
actorDelete(K_MYACTOR);
}
fprintf(stderr, "main thread is %d\n", threadSelf( ));
/* create N_THREADS new threads */
for(ind = 0; ind < N_THREADS; ind ++) {
result = newThread((KnPc) echoService, K_ACTIVE, "echoTh");
fprintf(stderr, "thread %d is created\n", result);
}
echoService( ); /* the main thread participates to the service */
}

c) An asynchronous client
In the following code, the client sends a message by calling ipcSend. The
corresponding command has two parameters:
- the rst one is the message that will be sent;
- the second one de nes the mode used for sending the message: if the rst
character is b the message is send in broadcast mode, otherwise it is sent in
functional mode.
--> cat $CHORUS/sources/ipc/service/client1/ipcClient1.c
#include "service.h"
KnCap groupCap; /* port group's capability */
KnPortLid portLid; /* port' local identifier */
int groupLid; /* group' local identifier */
int result; /* return values */
KnMsgDesc sndMsg; /* message descriptor for send */
KnMsgDesc rcvMsg; /* message descriptor for receive */
KnIpcDest destination; /* destination of message */
char message[BODY_SIZE]; /* buffer for receiving reply's body */
int mode; /* sending mode */

main(int argc, char *argv[ ]) {


if(argc != 3)
actorDelete(K_MYACTOR);
mode = (argv[2][0] == 'b') ? K_BROADMODE : K_FUNCMODE;
/* get capability of group corresponding to ECHO_STAMP */
if((groupLid = grpAllocate(K_STATUSER, &groupCap, ECHO_STAMP)) < 0) {
fprintf(stderr, "error on grpAllocate: %s\n",
strSysError(groupLid));
actorDelete(K_MYACTOR); }
/* prepare the destination address */
if((result = ipcTarget(&groupCap.ui, mode)) < 0) {
fprintf(stderr, "error on ipcTarget: %s\n", strSysError(result));
actorDelete(K_MYACTOR); }
390 Chapter 13. Communications
destination.target = groupCap.ui;
/* Prepare the message to be sent */
sndMsg.annexAddr = NULL; /* the message has no annex */
sndMsg.bodyAddr = (VmAddr) argv[1]; /* the body is the 1-st parameter */
sndMsg.bodySize = strlen(argv[1]) + 1 ;
/* The message is sent from the default port of the actor */
if((result = ipcSend(&sndMsg, K_DEFAULTPORT, &destination)) < 0) {
fprintf(stderr, "error on ipcSend: %s\n", strSysError(result));
actorDelete(K_MYACTOR);
}
fprintf(stderr, "ipcSend: %d\n", K_OK);
/* Prepare message descriptor to receive reply */
rcvMsg.bodyAddr = (VmAddr) message; /* address for the reply message */
rcvMsg.bodySize = BODY_SIZE; /* size of allocated space for reply */
rcvMsg.annexAddr = NULL; /* reply message has no annex */
/**************************** WARNING *********************************/
/* 2-nd parameter of ipcReceive is a pointer to a local identifier !!!! */
/************************************************************************/
portLid = K_DEFAULTPORT;
if((result = ipcReceive(&rcvMsg, &portLid, -1)) < 0) {
fprintf(stderr, "error on ipcReceive: %s\n", strSysError(result));
actorDelete(K_MYACTOR);
}
fprintf(stderr, "*** Client has received: %s\n", rcvMsg.bodyAddr);
}

d) Execution
 We rst try to execute the various clients while no server is available. The
rst is performed on a system where the IPC REMOTE feature is not active:
--> rsh neon arun cs -lM
started aid = 24
ChorusOS r4.0.0 Site 0 Time 4h 7m 32
............ IPC_L ............ (= local ipc
--> neon ipcClient1_u ABCDEFGHIJK f
started aid = 24
error on ipcSend: Unreachable destination
--> neon ipcClient1_u ABCDEFGHIJK b
started aid = 24
error on ipcSend: Unreachable destination
-->

We can check that whatever mode is used (functional or broadcast), a speci c


error value is returned and therefore the sender is advised that there is no
destination port.
The second test is performed on a system where the IPC REMOTE feature is
active:
13.3. The ChorusOS IPC 391
-1-> rsh neon arun cs -lM
.......... IPC_R .............. (= remote ipc
-1-> neon ipcClient1_u ABCDEFGHIJK f
started aid = 22
ipcSend: 0
(= calling thread is blocked
-2-> neon ipcClient1_u ABCDEFGHIJK b
started aid = 22
ipcSend: 0
(= calling thread is blocked
In that case, the ipcSend function does not detect an error and returns K OK;
in our example the thread is blocked on the ipcReceive call.
We will now use in our examples two systems neon and carbon within
the IPC REMOTE feature.
We start a server on carbon and an ipcClient1 client on neon. We can check
that the message is sent to the only port belonging to the group corresponding
to the service, and that this port is located on the carbon node:
-1-> carbon ipcServer_u
started aid = 2
main thread is 14
thread 9 is created
thread 12 is created
-Client-> neon ipcClient1_u ABCDE f
started aid = 2
ipcSend: 0

++ thread 14 has received a message


size = 6
body: ABCDE

*** Client has received: Initial value


-Client-> neon ipcClient1_u 1111111 b
started aid = 2
ipcSend: 0

++ thread 9 has received a message


size = 8
body: 1111111

*** Client has received: ABCDE


-Client->
 We now create a second server on the neon machine before executing di erent
clients. If a client is started on the neon node, the message is now sent to the
newly created port belonging to the group because it is located on the same site
392 Chapter 13. Communications
as the client (neon); the system always tries to send messages locally. A client
executing on carbon will send its message to the port located on carbon:
-2-> neon ipcServer_u
started aid = 2
main thread is 14
thread 9 is created
thread 12 is created
-Client-> neon ipcClient1_u 222222222 f
started aid = 22
ipcSend: 0

++ thread 14 has received a message


size = 10
body: 222222222

*** Client has received: Initial value


-Client-> carbon ipcClient1_u 3333333 f
started aid = 22
ipcSend: 0

++ thread 12 has received a message


size = 8
body: 3333333

*** Client has received: 1111111


-Client-> neon ipcClient1_u 444444 b
started aid = 22
ipcSend: 0

++ thread 14 has received a message


size = 7
body: 444444
++ thread 9 has received a message
size = 7
body: 444444

*** Client has received: 222222222


-Client-> carbon ipcClient1_u 5555 f
started aid = 22
ipcSend: 0
++ thread 9 has received a message
size = 5
body: 5555

*** Client has received: 444444


-Client->
13.3. The ChorusOS IPC 393
 Finally, we stop the service on the neon machine and starts a client on that
node. It sends its message to the only port belonging to the group and which
is located on the carbon node:
-Client-> rsh neon akill 2
-Client-> neon ipcClient1_u 66666 f
started aid = 2
ipcSend: 0

++ thread 12 has received a message


size = 6
body: 66666

*** Client has received: 5555


-Client->

13.3.7 Synchronous communications


13.3.7.1 Overview
Another mode for communicating by messages is a transactional one which cor-
responds to a remote procedure call (RPC). In this mode the sender is blocked
until it receives a response. Furthermore, RPC communications have a at most
once semantics; this means that an operation will never be performed more
than once and guarantees that the response received by a client relates to that
request. Finally, the protocol supports the propagation of abortion (see 8.8)
which means that if a client thread is blocked while waiting for an ipc reply,
abortion is propagated to the server thread that is currently servicing the re-
quest.
13.3.7.2 Initiating an RPC
An initial message is sent in RPC mode by calling the function
#include <ipc/chIpc.h>
int ipcCall(
KnMsgDesc *requestMsgDesc,
KnPortLid srcPortLi,
KnIpcDest *requestDest,
KnMsgDesc *responseMsgDesc,
int delay
);

In this call
394 Chapter 13. Communications
 srcPortLi is the local identi er of the source port for sending the message and
receiving the answer. As usual, K DEFAULTPORT can be used;
 delay is a timeout for the operation and is expressed as a number of milli-
seconds;
 requestMsgDesc points to the structure describing the request message to be
sent;
 requestDest points to the structure describing the destination of the message.
The various elds of the structure play the same role as their corresponding
arguments of the ipcSend function. It is important to note that K BROADMODE
addressing is forbidden for RPC;
 responseMsgDesc points to the structure describing the memory block where
the answer should be received. The elds are interpreted in the same way as the
elds of the corresponding arguments of the ipcReceive primitive. It is impor-
tant to note that the message received as answer does not become the current
message for the thread. This means that ipcReply, ipcSave, ipcGetData and
ipcSysInfo cannot be applied to the reply to the call.
The following negative values may be returned when an error occurs:
return value error type
K EABORT calling thread was aborted and message could not reach destination
K BADMODE broadcast mode forbidden for RPC
K EFULL communication system is saturated or destination queue is full
K EINVAL incorrect port local identi er
K EUNKNOWN incorrect group port identi er
K EFAULT address out of address space
K TOOMUCH body size too large
K ETIMEOUT timeout occurred
We now give the source code of a command for initiating an rpc transaction
towards a given destination port. The corresponding command will be passed
the unique identi er of the destination port as rst and second arguments,
the annex of the message to send as third argument and its body as the last
argument:
--> cat $CHORUS/sources/ipc/msgCall/msgCall.c
#include <stdio.h>
#include <chorus.h>
#include <ipc/chIpc.h>
KnCap actorCap;
KnUniqueId portUid;
KnIpcDest dest;
KnMsgDesc sndMsg, rcvMsg;
char annex[K_CMSGANNEXSIZE], char buffer[1024];
13.3. The ChorusOS IPC 395
main(int argc, char *argv[ ]) {
int result;
printf("Local identifier of thread calling ipcSend: %d\n", threadSelf( ));
sscanf(argv[1], "%x", &result);
portUid.uiHead = result;
sscanf(argv[2], "%x", &result);
portUid.uiTail = result;
/* preparing destination descriptor */
dest.target = portUid;
/* preparing message descriptors */
/* bzero(annex, K_CMSGANNEXSIZE) */;
strcpy(annex, argv[3]);
sndMsg.annexAddr = (VmAddr) annex;
sndMsg.bodyAddr = (VmAddr) argv[4];
sndMsg.bodySize = strlen(argv[4]) + 1;
rcvMsg.flags = 0;
rcvMsg.annexAddr = (VmAddr) annex;
rcvMsg.bodyAddr = (VmAddr) buffer;
rcvMsg.bodySize = 1024;
result = ipcCall(&sndMsg, K_DEFAULTPORT, &dest, &rcvMsg, -1);
if(result < 0)
fprintf(stderr, "error on ipcSend: %s\n", strSysError(result));
else
fprintf(stderr, "result = %d\n", result);
}

13.3.7.3 Sending a reply


From the point of view of the actor to which the initial message of an RPC is
sent,
 a thread of the actor will read the message by calling ipcReceive;
 a thread which can be expected to be the same one (but we can imagine it is
not) will send the reply by calling the ipcReply function we presented above.
13.3.7.4 Example
We can now give the synchronous version of the echo client (the ipcServer
server we developed in section 13.3.6.5 can serve this type of client as well as
asynchronous ones):
--> cat $CHORUS/sources/ipc/service/client2/ipcClient2.c
#include "service.h"
KnPortLid portLid; /* port local identifier */
int groupLid; /* group local identifier */
KnMsgDesc sndMsg; /* message descriptor for send */
KnMsgDesc rcvMsg; /* message descriptor for receive */
KnIpcDest destination; /* destination of message */
char message[BODY_SIZE]; /* buffer for receiving reply's body */
396 Chapter 13. Communications
main(int argc, char *argv[ ]) {
KnCap groupCap; /* port group capability */
int result, mode; /* return values and communication mode*/
if((groupLid = grpAllocate(K_STATUSER, &groupCap, ECHO_STAMP)) < 0) {
fprintf(stderr, "error on grpAllocate: %s\n",
strSysError(groupLid));
actorDelete(K_MYACTOR);
}
switch(argv[2][0]) {
case 'u': mode = K_FUNCUMODE;
sscanf(argv[3], "%p", &result);
destination.coTarget.uiHead = result;
sscanf(argv[4], "%p", &result);
destination.coTarget.uiTail = result;
break;
case 'x': mode = K_FUNCXMODE;
sscanf(argv[3], "%p", &result);
destination.coTarget.uiHead = result;
sscanf(argv[4], "%p", &result);
destination.coTarget.uiTail = result;
break;
default: mode = K_FUNCMODE;
}
if((result = ipcTarget(&groupCap.ui, mode )) < 0) {
fprintf(stderr, "error on ipcTarget: %s\n", strSysError(result));
actorDelete(K_MYACTOR);
}
destination.target = groupCap.ui;
/* Preparation of the message to be sent */
sndMsg.annexAddr = NULL; /* the message has no annex */
sndMsg.bodyAddr = (VmAddr) argv[1]; /* the body is the 1-st parameter */
sndMsg.bodySize = strlen(argv[1]) + 1;
/* Preparation of message descriptor for receiving reply */
rcvMsg.bodyAddr = (VmAddr) message; /* address for the reply message */
rcvMsg.bodySize = BODY_SIZE; /* size of allocated space for reply */
rcvMsg.annexAddr = NULL; /* reply message has no annex */
/* a rpc is sent from the default port of the actor */
if((result = ipcCall(&sndMsg, K_DEFAULTPORT, &destination,
&rcvMsg, -1)) < 0) {
fprintf(stderr, "error on ipcCall: %s\n", strSysError(result));
actorDelete(K_MYACTOR);
}
fprintf(stderr, "*** Client has received: %s\n", rcvMsg.bodyAddr);
}

The following traces illustrate the use of di erent functional addressing modes.
The rst sequence is similar to the rst one executed with ipcClient1 clients:
13.3. The ChorusOS IPC 397
-1-> carbon ipcServer_u
started aid = 2
main thread is 14
thread 9 is created
thread 12 is created

Client-> neon ipcClient2_u AAAAAA f


*** Client has received: Initial value
++ thread 14 has received a message
size = 7
body: AAAAAA

started aid = 2
-Client->

-2-> neon ipcServer_u


started aid = 2
main thread is 14
thread 9 is created
thread 12 is created

-Client-> neon ipcClient2_u BBB f


started aid = 22

++ thread 14 has received a message


size = 4
body: BBB

*** Client has received: Initial value


-Client->

We retrieve the unique identi ers of the ports belonging to the server on the
carbon node:
-Client-> rsh carbon arun cs -la 2
started aid = 22
................................
PORT-UI PORT-LI .....
20000016 869da806 def ........
40000001 869da806 0000 .......
................................
-Client->
We rst execute a client on neon using the associative functional mode. It
requests that its message be sent to a port belonging to the group and located
on the same node as the object whose unique identi er is "20000016 869da806"
(it is the default port of the server running on carbon). Thus the message is
sent to that server:
398 Chapter 13. Communications
-Client-> neon ipcClient2_u CC u 20000016 869da806
started aid = 22

++ thread 9 has received a message


size = 3
body: CC

*** Client has received: AAAAAA


-Client->

The last client is executed on carbon and uses the exclusive functional mode.
It requests its message to be sent to a member of the group whose unique
identi er is di erent from "40000001 869da806" which is the port's UI of the
server running on carbon. Therefore, the message is sent to the server running
on neon:
-Client-> carbon ipcCLient2_u DD x 40000001 869da806
started aid = 22
++ thread 12 has received a message
size = 3
body: DD
*** Client has received: BBB
-Client->

13.3.8 Manipulating the current message


The next gure summarizes the evolution of the current message of a given t1
thread according to di erent events occurring within that thread:
After creation of the thread t1 :
t1 has no current message and no saved message
p1 p2 After t1 has received a message m1 on p1 or p2 by calling ipcReceive:
m1 is the current message of t1
After t1 has saved m1 by executing idMsg1 = ipcSave(K NONEPORT)
t1 has no current message and a saved message identi ed by idMsg1
After t1 has received a message m2 on p1 or p2 by calling ipcReceive:
m2 is the current message of t1
After t1 has saved m1 by executing idMsg2 = ipcSave(K NONEPORT)
t1 has no current message and 2 saved messages idMsg1 and idMsg2
After t1 has restored m1 by executing ipcRestore(idMsg1)
t1 m1 is the current message of t1 and t1 has a saved message idMsg2
After t1 has replied by calling ipcReply
t1 has no current message and a saved message identi ed by idMs