Efficient Distributed Shared Memory on a Single System Image Operating

System

Jesús De Oliveira Jesús Federico Rafael Chacón David Zaragoza

Yudith Cardinale
Universidad Simón Bolı́var,
Departamento de Computación y Tecnologı́a de la Información,
Apartado 89000, Caracas 1080-A, Venezuela

{jdeoliveira,jfederico,rchacon,dzaragoza,yudith}@ldc.usb.ve

Abstract Message passing systems typically use explicit data dis-

In this paper we describe KRC, a kernel-level imple-
mentation of DSM paradigm using the Release Consistency
(RC) Memory Model on the Kerrighed Single System Image
operating system. KRC offers a write-update coherence pol-
icy and a Multiple Reader/Multiple Writer (MR/MW) data
access algorithm. The main contribution of this work is
an adaptive update mechanism which efficiently implements
the write-update protocol and the MR/MW data access pol-
icy. It consists on the local replication of shared pages
(twin copies) and encoding dynamically the changes made
on the original versions, hence, only the differences (diffs)
are sent to processes sharing data instead of the full modi-
fied pages. We present an experimental evaluation of KRC
compared with Kerrighed in terms of total execution time,
which shows that the execution of parallel applications on
KRC, in particular those with false sharing, has a signifi-
cant performance improvement over the original Kerrighed
platform. We also show that KRC provides linear scalabil-
ity.
1 Introduction
Clusters are considered today as a standard platform for
high performance computing, and have supported parallel
applications processing for many years, primarily through
two main parallel computing models: the Message Pass-
ing Paradigm, targeted at providing a standard and homo-
geneous distributed communication environment to coor-
dinate processes running across the nodes of the comput-
ing cluster; and the Distributed Shared Memory (DSM) ap-
proach, oriented at providing a single, virtualized, parallel
environment on top of a distributed clustered architecture.
tribution, exchange, and synchronization. These systems
are typically easier to construct but more difficult to pro-
gram; the inverse is true for their shared memory coun-
terparts. Programming parallel applications with a shared
memory model is considered to be more natural and easier
than programming with a message-passing model.
DSM supports the convenient shared-memory program-
ming model on distributed-memory hardware. Implement-
ing a Single System Image (SSI) is an elegant approach to
simplify global resources managed in a cluster [4, 5, 10,
13, 21]. The SSI concept hides the heterogeneous and dis-
tributed nature of the available resources, in order to present
a global and uniform view to user and applications. Algo-
rithms for implementing DSM deal with two basic prob-
lems: distributing data to minimize the latency and main-
taining a coherent view of data without incurring in high
overhead.
To deal with the problem of distributing shared data
across the system, there are two frequently used policies:
Single Writer (SW) and Multiple Writer (MW) [2, 19]. The
SW approach means that only one process has rights to
modify a shared data item. This policy can be implemented
in the form of Single Writer/Single Reader (SW/SR), and
more commonly in the form of Single Writer/Multiple
Reader (SW/MR) algorithm. At any given time, data may
be either read in read-only mode by one or more processes
or written by a single process. On the other hand, the MW
approach means that several processes have rights to update
a shared data item simultaneously. This policy is commonly
implemented in the form of MW/MR algorithm.
For keeping data coherence, two possibilities exist:
write-update or write-invalidate protocols. In the update-
based protocol, modifications made by a process are exe-
cuted locally and immediately forwarded to all other pro-
cesses holding a copy of the modified data item, effectively
updating it. Hence, processes read their local copies without
the need for communication. In the invalidate-based proto-
col, updates made by a process are executed locally and an
invalidating message is sent to all other processors holding
a copy of the data item, which immediately invalidate the
data read by local processes. Then, when local processes
access the invalidated data item, a read miss occurs and a
valid copy of the data item (the most recent) will be fetched
from the writing processor.
A Memory Consistency Model defines how applications
use the shared address space (i.e., the legal ordering of
memory references issued by a processor), whereas the
degree of relaxation of a consistency protocol determines
the efficiency of the implementation [11, 16, 17, 18, 20].
Stronger forms of the consistency model (i.e., strict and se-
quential consistency models) typically increase the memory
access latency and bandwidth requirements, while simplify-
ing programming. Relaxed models (i.e., weak, release, and
lazy release consistency models) are based on looser con-
straints. In order to reduce communication overhead caused
by writes propagation, these models allow multiple copies
of shared data to exist on different nodes at the same time,
some of which can be temporarily inconsistent. They distin-
guish ordinary synchronization memory accesses executed
between acquire-release operations pairs. Then, memory
consistency is required only on these synchronization ac-
cesses.
In this paper we describe a kernel-level implementation
of the Release Consistency (RC) Memory Model, with a
MW/MR data access policy and a write-update protocol.
Our implementation extends Kerrighed SSI operating sys-
tem [13], for System-V shared memory IPC mechanisms.
We called KRC (Kerrighed-Release Consistence) the ex-
tended version of Kerrighed. The RC Memory Model im-
plemented in KRC is efficient, adaptive, and transparent.
It efficiently implements the write-update protocol and the
MW/MR data access policy, providing an adaptive update
mechanism to transfer the changes among the processes
which share data. This adaptive update mechanism is our
main contribution and consists on locally replicating the
shared pages (twin copies generation) and dynamically en-
coding the changes made on the original pages employing
several byte-encoding schemes. Hence, only the differences
(diffs) are sent to processes sharing data, instead of whole
modified pages. KRC also provides transparent execution to
applications, i.e., no instrumentation/modification is needed
for shared memory parallel applications to be executed on
clusters platforms. We present an experimental evaluation
of KRC, compared with Kerrighed in terms of total execu-
tion time by using confidence intervals for the means. The
results show that the execution of parallel applications on
KRC, especially those with false sharing, has a significant
performance improvement over the original Kerrighed plat-
form. We also show that KRC presents a linear scalability.
2 Overview of Kerrighed
Kerrighed [13] is a SSI operating system based on
Linux, on which every node of a cluster has a consolidated
view of available system resources and constructs. Ker-
righed is based on a global, kernel-level, memory man-
agement service called kernel Distributed Data Manager
(kDDM). The kDDM allows to share collections of objects
among the cluster nodes, which are stored on one node and
can be shared and accessed from any other node in a trans-
parent fashion. This way, the kDDM enables transparent
cluster-wide storing and sharing of memory segments (e.g.,
memory pages) through the standard System-V SHM mech-
anisms, treated abstractly as objects.
The data in the kDDM is hierarchically organized by ob-
jects, sets, and namespaces. The objects are managed by the
kDDM without any assumptions on its contents or seman-
tics: in terms of Kerrighed, objects are a set of bytes defined
by the programmer. The objects can represent basic types
(integers, characters, etc.) as well as memory pages, files,
devices, and custom data structures, allowing developers to
transparently share any type of data between nodes. An ob-
ject must be always associated with a set, which stores a
collection of similar objects. On the top level of the hier-
archy, a namespace acts as a collection of sets. In general,
every kDDM object belongs to a set, and sets in turn belong
to a namespace.
The objects are handled through manipulation functions
that ensure cluster-wide data replication and guarantee con-
sistency. These functions can be compared with standard
read/write locks. They are primarily used to define critical
sections inside the kDDM, allowing secure and consistent
access to objects. As objects can be replicated among
cluster nodes, each object in the kDDM is associated
to a state, which can be READ COPY and INV COPY,
among the most significant ones. The READ COPY state
represents that a replica of the object could be read locally,
while the INV COPY state means that the local replica has
been updated on another cluster node and the local copy
is outdated. The objects manipulation interface mainly
consists of these operations:
get: Equivalent to the acquisition of a read-lock upon a
shared object. This operation replicates the object in the
local memory region of the requester node, and marks it
ensuring that no write operations can be performed on the
rest of the nodes, however, read operations are permitted.
grab: Equivalent to the acquisition of a write-lock upon
a shared object. It works as the get operation, but also
ensures that read operations are prohibited in the rest of the
nodes. When a grab is performed, every replica of the
object present on other nodes is invalidated.
put: Equivalent to a release of the locks associated to a
shared object. It notifies the other nodes that, from now on,
the object is no longer in use at the local node.
From the above, we can realize that kDDM provides
a strict consistency model to shared objects with an
invalidate-based coherence policy and a MR/SW data ac-
cess algorithm.
3 KRC: Implementation of Release Consis-
tency on Kerrighed
According to extensive studies and experimental re-
sults [1, 11, 16, 17, 18, 20] relaxed consistency models have
proven increased efficiency compared to strict consistency
models, because of the remarkable savings of communica-
tion by reducing the number of messages and the amount
of data exchanged for remote memory access. Hence, we
decided to implement Release Consistency (RC), a relaxed
consistency model and a MW/MR algorithm.
The RC model establishes that memory is consistent only
at specific synchronization points. It is based on the fact
that programmers use synchronization points to separate ac-
cesses to shared variables by different processes, for exam-
ple by enclosing the shared data in critical sections. Due
to the fact that these primitives ensure that inside a criti-
cal section no other process concurrently accesses a specific
shared data item, it could be temporarily inconsistent. Syn-
chronization points are defined through two operations upon
shared objects: Acquire, which implies the start of a critical
section on which shared objects are manipulated, and Re-
lease, which represents the finalization of the critical sec-
tion. On the other hand, MW/MR algorithm for parallel ap-
plications based on critical sections, has better performance
than SW/SR and SW/MR, by providing a higher degree of
concurrency. Additionally, the choice of a MW/MR algo-
rithm forces the use of the write-update protocol instead of
the write-invalidate protocol, which could cause loss of up-
dates. Due to high data locality, it is possible that on a given
time, concurrent processes could be accessing different data
items located at the same page. Using a write-update proto-
col, all these processes will get every modification made by
the other processes at the end of their corresponding critical
section. On the contrary, using the write-invalidate proto-
col, the first process that finishes its critical section invali-
dates all copies of the memory page, causing modifications
performed by the other concurrent processes to be lost.
3.1 Extending Kerrighed
KRC (Kerrighed-Release Consistency) is the result of in-
corporating into Kerrighed the RC model with a MW/MR
data access algorithm and a write-update coherence policy.
In order to provide DSM capabilities, we performed modi-
fications at the kernel-level SHM management routines, in-
cluding System-V SHM and IPC semaphores system calls
(e.g., shmget, shmat, shmctl, shmdt, sem open,
sem close, sem wait, and sem post).
Through the kDDM, when the cluster is initialized, a set
is instantiated to store information about processes execut-
ing inside a critical section. This set is referred as Criti-
cal Process Container (CPC). As any other Kerrighed ob-
ject, this set is shared among cluster nodes, and its consis-
tency is preserved under Kerrighed standard strict consis-
tency model.
The RC model implemented in KRC is divided in
three phases related to the natural sequence of operations
performed by a process during the execution of a critical
section. The first phase corresponds to the actions carried
out when an acquire operation is performed to enter to
a critical section. The second phase describes how the
protocol manages data structures to keep memory in a
consistent state. Finally, the third phase corresponds with
the actions carried out when a release operation is executed,
signalling the exit of the critical section. These phases are
illustrated in Figure 1.
Figure 1. 3-phases RC in KRC.
Acquire: When a process acquires a semaphore, it is
registered on the CPC, representing that it has entered a
critical section. At this time a new set structure, called
Modified Pages Set (MPS), which will contain pointers
to the SHM pages modified by the current process, is
associated with the process. Figure 2 shows the CPC before
and after a process k executes an acquire.
Critical Section: When a process triggers a write-fault
inside a critical section, a new object is inserted in the
corresponding MPS. Before giving the process write access
to the page (by allocating a local replica) a page twin is
created. This twin will be used to determine the changes
made to the page in the critical section and notify the set of
nodes that contain replicas (copy set) about those changes.

Figure 2. Acquire example in KRC.
This notification occurs at release time.
Release: When a process reaches a release operation, it is
necessary to send a notification to the copy set along with
the changes made on the shared pages. In order to iden-
tify the changes, each page in the MPS is compared with its
corresponding twin to obtain the differences (diffs). Hence,
only the diffs are sent to the copy set, instead of whole
pages. This phase implements the adaptive update mech-
anism, which is presented in Section 3.2.
3.2 The KRC adaptive update mechanism
The adaptive update mechanism is our main contribu-
tion and consists on locally replicate the shared pages (twin
copies) and dynamically encoding the changes made on the
original ones, by using several byte-encoding schemes.
To encode the diffs, the protocol divides a memory page
in blocks of 4, 8, 16, 32, or 64 bytes (i.e., a 4KB page is
divided in 1024, 512, 256, 128, or 64 contiguous blocks
respectively). Then, for each block it is determined which
bytes were changed and with those, the update message is
built, which includes the following information: i) block
size identifier, composed of 3 bits; ii) block number identi-
fier, composed of 6 to 10 bits, according to the block size
used, for each modified block; iii) new bytes of the block.
The smallest block size for a update message is dynam-
ically determined in order to accommodate all the memory
page updated bytes. In this sense, the calculation is per-
formed by generating 5 update messages simultaneously,
each using a different block size. The smallest one is then
selected to be sent to the copy set. The creation and main-
tenance cost of twins are compensated by reduced network
traffic and smaller notification message size. Our adaptive
diffs encoding mechanism, shown in Fig. 3, guarantees a
minimal-sized update message.
In contrast to the original Kerrighed implementation,
where an object owner is the last node which executed a
grab operation on it, in KRC the object owner is the node
that creates it. The owner has the responsibility to inform
the copy set each time a new node becomes a member of
this set. In this way, every node has the copy set updated. In
turn, when a node first requests a copy of an object through
Figure 3. Adaptive Update Mechanism in KRC
a grab or get operation, this request is handled by the
owner.
In order to identify nodes that holds a write replica of
an object, we introduced the WRITE COPY state. Nodes
that require write access to pages locally replicated, can go
directly to the WRITE COPY state without notice. This is
possible because of the write-update protocol: as an object
owner is responsible for managing the object copy set, an
object in this state can be written on any node under the
premise that it is being accessed in a critical section and it
will not be written simultaneously by more than one pro-
cess. Hence, on release time, it is ensured that the cor-
responding node will notify every replica holder about the
modifications made by the current process according to the
steps described in Section 3.2. Figure 4 shows a simplified
state diagram including the new WRITE COPY state. Note
that a release operation triggers implicit transitions from
WRITE to READ states. This way, after a release opera-
tion, subsequent writes will trigger a page write-fault and
will cause the proper registration of that page on the MPS.
This avoids dirty pages and lost updates.
Figure 4. Diagram of the state transitions to
keep data coherence in KRC
4 Experimental results
In order to evaluate the performance of KRC, we con-
ducted two experimental scenarios with three synthetic ap-
plications: i) a comparative analysis, in which we executed
test applications with fixed problem sizes and fixed num-
ber of nodes, on Kerrighed and on KRC systems; ii) a KRC
scalability analysis, in which we run the test applications
with fixed problem sizes (higher than those used on the first
scenario), varying the number of nodes.
 Our applications exploit different levels of data locality.
The first application consists on a simple Monte-Carlo sim-
ulation to approximate the value of π which involves ran-
domly selecting N points in the unit square and determin-
ing which of these points are in the circumscribed circle.
In our version, all processes select N points accessing the
same unit square (this data is located in one page, shared by
the processes). The node P 0 is responsible to collect partial
results and report the final approximate value of π.
The second application is an implementation of Riemann
Sums integration which distributes, in an equitable manner,
the limits of the integral among the number of running pro-
cesses. A coordinator process is responsible for adding the
results of the rest of the processes. The integral used in our
R 12 1 false-sharing phenomenon occurrence on high data locality
experiments was 0 (x3 + 3(x2 ) + x + 2)dx .
2 applications. The matrix multiplication application does not
Finally, the third application consists on a traditional par-
allel application for matrix multiplication, which distributes
the calculation among all running processes. Each process
computes a sub-part of the multiplication.
Applications were implemented using System-V SHM
inter-process communication mechanism. We performed 10
executions for each test case, running one process per com-
puting node and measuring the wall clock execution time.
4.1 Discussion of the Comparative Analy-
sis
on KRC perform better than when they are executed on the
original Kerrighed implementation.
The poor performance of Kerrighed when compared
with KRC, specially on high data locality applications, such
as Monte-Carlo π estimation and Riemann sum integra-
tion, is a consequence of the phenomenon known as false-
sharing. Due to the page granularity of the shared mem-
ory, large amounts of data are located on the same page.
Hence, a flood of invalidation messages and unnecessary
full-content page transfers with a ping-pong behaviour oc-
cur when different processors request write access to the
same page, even when each of them write on a different
data item. In contrast, the KRC MW/MR access policy
avoids the ping-pong effect and completely eliminates the
present the phenomenon of false-sharing: in this case, the
results show that, with KRC, we obtain similar performance
to that with Kerrighed.
From these results, we expect that KRC performs much
better than Kerrighed on real applications with a high de-
gree of data locality, when false-sharing can appear, or sim-
ilar to Kerrighed in other cases.
4.2 Discussion of the KRC Scalability
Analysis

The comparative analysis experiment was executed on The scalability analysis experiment was executed on a
a dedicated 12-nodes cluster running Debian GNU/Linux dedicated 32-nodes cluster running Debian GNU/Linux 5.0,
4.0rc5, dual core 3.4Ghz Pentium IV CPU, 1GB of RAM dual core 3.4Ghz Pentium D CPU, 4GB of RAM and inter-
and interconnected trough a 100Mbps Ethernet LAN. Table connected through a 1Gbps Ethernet LAN.
1 shows average wall clock execution time of each experi-
ment.
Application Problem size KRC Kerrighed Relative difference
Matrix multiplication 128x128 matrix size 0.78 s. 0.75 s. -3.85%
256x256 matrix size 1.11 s. 1.39 s. 20.14%
512x512 matrix size 1.84 s. 1.82 s. -1.09%
1024x1024 matrix size 8.18 s. 10.47 s. 21.87%
2048x2048 matrix size 63.58 s. 65.11 s. 2.35%
Montecarlo π estimation 1 ∗ 104 points 0.81 s. 0.78 s. -3.7%
1 ∗ 105 points 0.83 s. 7.42 s. 88.81%
5 ∗ 105 points 0.81 s. 19.51 s. 95.85%
5 ∗ 106 points 1.59 s. 212.80 s. 99.25%
25 ∗ 106 points 4.49 s. 1132.07 s. 99.6%
Riemann sum integration 1/102 precision 0.79 s. 0.64 s. -18.99%
1/104 precision 0.80 s. 0.75 s. -6.25%
1/105 precision 0.82 s. 4.04 s. 79.7%
1/106 precision 0.82 s. 33.00 s. 97.52%
1/107 precision 1.94 s. 301.03 s. 99.36%
Relative diff. average 44.71%
Relative diff. standard deviation 49.22%
Confidence interval [2.43, 86.98]
(α = 0.05)

Figure 5. KRC scalability

Table 1. Confidence intervals analysis of
comparative experiment The results presented on Figure 5 show that the kernel-
level adaptations performed in Kerrighed in order to imple-
ment the new consistency model have no impact on appli-
From the statistical analysis presented on Table 1, we can cations performance scalability. The result of increment-
argue with 99.5% of confidence that applications executed ing the number of computing nodes used, and hence the
number of concurrent processes, presents a practically lin-
ear speed-up for applications designed with a divide-and-
conquer strategy, such as Riemann sum integration and ma-
trix multiplication. On the other hand, the Monte-Carlo
π estimation shows no performance degradation when the
number of local replicas of shared memory pages, among
which consistency must be guaranteed, increases as the
number of processes is raised.
Note that without the optimized communication mech-
anisms implemented in KRC and the MW/MR data access
algorithm described in section 3, an increase of the com-
puting nodes and hence the concurrent executing processes
would cause a tremendous increment on inter-process com-
munication activities in Kerrighed. This is due to a higher
number of processes generating full-content page updates
upon each process write to the shared memory pages. In this
case, network bandwidth will become a bottleneck for ap-
plications performance much faster than on KRC, resulting
in a hard constraint to applications performance scalability,
as occurs on the original Kerrighed implementation.
5 Related Work
The first implementations of DSM have shown the trade-
off between flexibility and performance for sharing and ac-
cessing data by comparing conservative consistency models
(such as that implemented in Ivy [14]) with relaxed con-
sistency models (such as those implemented in Munin [3]
and TreadMarks [12]). JiaJia[7] presents an intermediate
relaxed consistency model located between entry consis-
tency and release consistency, namely scope release con-
sistency, on which writes are propagated only to processes
on the same scope, defined by the critical section protection
constructs. Performance results show a significant reduc-
tion of communication messages, leading to better appli-
cations performance. More recent improvements over the
scope consistency are presented in JUMP[6], which slightly
modifies the protocol and consistency model implemented
in JiaJia, to allow migrating memory segment among nodes,
hence adapting dynamically to application memory access-
pattern and reducing communication. Our proposal imple-
ments a release consistency model, equivalent to that pre-
sented in Munin [3]. All these early DSM systems provide
a user level library or API used to declare and use shared
memory segments on their applications, then they are not
fully transparent; while with KRC it is not required to alter
applications source code in order to take advantage of DSM
benefits.
The combination of SSI platforms and DSM mech-
anisms has proven to be an attractive environment for
straightforward application development and high perfor-
mance computing, providing the user with a fully trans-
parent view of a cluster of PCs as a unique, consolidated
computing resource. However, few successful implemen-
tations of such systems exist. Plurix [10] creates a glob-
ally shared memory space, on which all resources, oper-
ating system kernel, drivers, applications code, and data
are stored. Consistency is guaranteed by a transactional
consistency model, based on optimistic transaction consis-
tency models found on common database systems. Ker-
righed [13] implements sequential consistency model. Per-
formance results show similar speedups of equivalent par-
allel applications running on Plurix and Kerrighed [8]. As
we presented in the experimental results section (Section 4),
Kerrighed performance with sequential consistency is over-
stepped by our relaxed consistency implementation. Team-
ster [4] provides a Global Memory Image that allows easy
threads and processes distribution and migration accom-
plished in a fully transparent fashion. Unlike our proposal,
on which applications must employ System-V SHM mecha-
nism to take advantage of distributed shared memory, Team-
ster provides a fully transparent SSI platform. Static global
variables are allocated from a sequential consistent memory
segment, and dynamic heap is allocated from a release con-
sistent memory area. The main difference with our proposal
is that Teamster platform is not an open source project.
Jessica [21] provides a single Java virtual machine image
that allows transparent distributed execution of Java appli-
cations. It employs the release consistency or lazy release
consistency model to support consistency in the Global Ob-
ject Space. KRC has no restrictions over the programming
language of applications. vNuma [5] provides NUMA ar-
chitectures virtualization on top of a cluster of PCs. It im-
plements the SW/MR data access policy and the sequential
consistency model. Experimental results show poor perfor-
mance compared to clustered based applications counter-
parts, but the authors argue that in future work, overall per-
formance of the platform will be addressed.
More recently research and experimentation efforts have
been focused on extending the DSM approach to cluster
federations or computing grids. These proposals must deal
with very high-latency/low-bandwidth and highly dynamic
communication networks. Hence, reducing communication
overhead for consistency maintenance, as well as address-
ing fault tolerance, topology change, and resource discov-
ering are the main priorities on grid-aware DSM platforms.
Among the first proposals of such systems are JuxMem[9]
and Teamster-G[15]. We plan to incorporate grid-awareness
and automatic data distribution mechanisms considering
communication costs to KRC in the near future.
6 Conclusions
The implementation of the consistency model is the key
issue when dealing with DSM systems. The SSI operating
system introduced in this work, called KRC, implements
the Release Memory Consistency Model on Kerrighed. Our
experimental results showed that KRC performs much bet-
ter than Kerrighed, especially when the phenomenon called
false-sharing can appear. Besides, KRC offers a transpar-
ent DSM to applications, and thus facilitates parallel ap-
plication programming since the shared memory program-
ming model is often more natural than the message-passing
paradigm. We are currently enhancing KRC, in order to
support adaptive protocols according to the application be-
haviour. We also plan to incorporate grid-awareness and
automatic data distribution mechanisms into KRC.
References
[1] C. Amza, A. L. Cox, S. Dwarkadas, L.-J. Jin, K. Rajamani,
and W. Zwaenepoel. Adaptive protocols for software dis-
tributed shared memory. In Proc. of IEEE, Special Issue on
Distributed Shared Memory, pages 467–475, March 1999.
[2] C. Amza, A. L. Cox, W. Zwaenepoel, and S. Dwarkadas.
Software dsm protocols that adapt between single writer and
multiple writer. In Proc. of the 3rd IEEE Symposium on
High-Performance Computer Architecture (HPCA), 1997.
[3] J. K. Bennett, J. B. Carter, and W. Zwaenepoel. Munin:
distributed shared memory based on type-specific memory
coherence. SIGPLAN Not., 25(3), 1990.
[4] J. B. Chang and C. K. Shieh. Teamster: A transparent dis-
tributed shared memory for cluster symmetric multiproces-
sors. In Proc. of the 1st Int. Symposium on Cluster Comput-
ing and the Grid, pages 508–513, May 2001.
[5] M. Chapman and G. Heiser. vNUMA: A virtual shared-
memory multiprocessor. In Proc. of the 2009 USENIX An-
nual Technical Conf., pages 349–362, 2009.
[6] B. Cheung, C. Wang, and K. Hwang. Jump-dp: A soft-
ware dsm system with low-latency communication support.
In Proc. of the Internat. Workshop on Cluster Computing
- Technologies, Environments and Applications, pages 1–7,
June 2000.
[7] M. R. Eskicioglu and T. A. Marsland. Shared memory com-
puting on sp2: Jiajia approach. In Proc. of the 1998 Conf. of
the Centre for Advanced Studies on Collaborative research,
page 12, Nov. 1998.
[8] S. Frenz, R. Lottiaux, M. Schoettner, C. Morin, R. Goeck-
elmann, and P. Schulthess. A practical comparison of clus-
ter operating systems implementing sequential and transac-
tional consistency. Distributed and Parallel Comp., pages
23–33, Oct. 2005.
[9] A. Gabriel, B. Luc, and J. Mathieu. Juxmem: An adap-
tive supportive platform for data sharing on the grid. Scal-
able Computing: Practice and Experience, pages 45–55,
September 2005.
[10] R. Goeckelmann, M. Schoettner, S. Frenz, and P. Schulthess.
Plurix, a distributed operating system extending the single
system image concept. In Proc. of IEEE Canadian Conf. on
Electrical and Computer Engineering, pages 1985 – 1988,
2004.
[11] L. Hammond, B. D. Carlstrom, V. Wong, B. Hertzberg,
M. Chen, C. Kozyrakis, and K. Olukotun. Programming
with transactional coherence and consistency (tcc). ACM
SIGPLAN NOTICES, pages 1–13, 2004.
[12] P. Keleher, A. L. Cox, S. Dwarkadas, and W. Zwaenepoel.
Treadmarks: Distributed shared memory on standard work-
stations and operating systems. Proc. of the 1994 Winter
Usenix Conference, pages 115–131, January 1994.
[13] A. Lèbre, R. Lottiaux, E. Focht, and C. Morin. Reducing
kernel development complexity in distributed environments.
LNCS, 5168:576–586, 2008. Euro-Par.
[14] K. Li. Ivy: A shared virtual memory system for parallel
computing. In Int. Conf. Parallel Processin, pages 94–101.
IEEE Computer Society Press, 1988.
[15] T.-Y. Liang, C.-Y. Wu, C.-K. Shieh, and J.-B. Chang. A
grid-enabled software distributed shared memory system on
a wide area network. Future Generations Computer Sys-
tems, 23(4):547–557, 2007.
[16] P. Schmidt, S. Frenz, S. Gerhold, and P. Schulthess. Transac-
tional consistency in the automotive environment. In Proc.
of the Third International Symposium on Industrial Embed-
ded Systems, pages 1–4, 2008.
[17] H. Weiwu, S. Weisong, and T. Zhimin. An interaction of co-
herence protocols and memory consistency models in dsm
systems. Journal of Computer Science and Technology,
13(2):110–124, 1998.
[18] M. Wende, M. Schoettner, R. Goeckelmann, T. Bindham-
mer, and P. Schulthess. Optimistic synchronization and
transactional consistency. In Proc. of the 2nd IEEE/ACM
Int. Symposium on Cluster Computing and the Grid, 2002.
[19] X. Xianghui and H. Chengde. Limited multiple-writer: An
approach to dealing with false sharing in software dsms.
Journal of Computer Science and Tech., 15(5):453–460,
2000.
[20] Y. Zhou, L. Iftode, J. P. Singh, K. Li, B. R. Toonen,
I. Schoinas, M. D. Hill, and D. A. Wood. Relaxed con-
sistency and coherence granularity in dsm systems: A per-
formance evaluation. In Proc. of the 6th ACM Symposium
on Principles and Practice of Parallel Programming, vol-
ume 32, pages 193 – 205, 1997.
[21] W. Zhu, C.-L. Wang, and F. C.M.Lau. Jessica2: A dis-
tributed java virtual machine with transparent thread migra-
tion support. In Proc. of IEEE Fourth International Confer-
ence on Cluster Computing, pages 381–388, 2002.