Professional Documents
Culture Documents
Systems
Ahmed E<. Ezzat
He,wlett-PacEard Laboratories
1501 Page M i l l Rd.
Palo Alto, CA 94303
ezzat @hplabs.hp.com
4M
0 1990 IEEE
0-8186-2088-9/90/0000/0403/$01.00
in;ilt.io:? iii n distribute~ltransact~ioi~
cnvironnwnt: en- responsible for ensuring that transient incmsistencies in
ye7 algorit3hms [Herlihy 89, McICeiidry 861 which use the atomic components do not produce permanent in-
real-time clodis to ensure t,liat orphans are eliminated consistencies in the nonatomic components. An orphan
within a fixed durat,ion, lazy aIgorit,hms [Herlihy 89, elimination scheme based on Walker's method has been
McIiendry S6] which use !ogical clocks to ensure that implemented as part of the Argus system [Liskov 871 to
orphans are eventually eliminated as information prop- deal with that issue.
agates through the syst,em, and a topological change al-
This paper addresses orphan detection and eiimina-
gorithm suita.ble for dealing with orphans resulting from
net:work pa.rtit,ioning [Mueller 831 in a dist,ributed trans- tion for applications in a dist,rihut,edobject-oriented sys-
act,ion system. Muelier's algorithm is driven by both the tems. We apply our model for a distributed lJnix/C++
t,ransactionhome sites and tlie transaction synclironiza- environment., but the model is general and ca.n be ap-
plied to other distributed object-oriented systems. The
tion sites (TSS) for the different, files opened from the
1 ransaction.
paper is organized as follows. Section 2 describes the
background for this work and gives definitions for the
Outside t>h.etransaction doniain, t,he orphan elimina- terminology used in t,he paper. Section 3 presents the
tion problem was first identified by Birrell [Birrell 831, problem definition. Section 4 discusses our model. Sec-
and solutions based on timeouts have been proposed by tion 5 uses a simple example in a distributed Unix/C++
Lampson [Larnpson 811 and Ra.jdoot [Panzieri 881. De- enviroiiment, to describe our model in more detail, and
pending on h e RPC semant,ics in a dist,ributed system, we conclude hy a siimn-iary in Sect,ion 6.
additional requirements might) be needed for correct,ness
criteria. In particular, for ut niost once calls, not only
should orphans be aborted, t,he almtioii must include 2 Background and terminology
undoing of a.ny side effects that inay have been pro-
duced [Panzieri 881. Wallw [Wallter 841 has proposed a
In this section, we define sonie t r r n ~ sused by our model
transaction-based orphan elimina.tion scheme that dy-
in this paper.
namically tracks dependencies a.mong transactions. His
scheme uses timeouts to keep the information sent in
messages t,o a ma,nageable level.
2.1 Iiiiplenientatioii set
In a conveiitioiial transaction system, orphans have no
impact, on d a t a consistency, where a transaction does An implementmationset is an object with a well defined
not interact with the outside world until it commits. interface representing the traditional Unix process ab-
However, in a general purpose distributed system, such straction in an object-oriented system environment. An
inconsistencies may he more prohlema.tic. For exam- iniplementation set contains many objects (persistent
ple, the Argus system [Liskov S3] supports a method- a.nd non-persistent), and exports operations to admin-
ology in which user-defined a.tornic data, types are ini- ister these objects as well as control migrating objects
plernent,ed by a mixture of atomic and nonatomic data between different implementation sets. Implementat'ion
types at a lower level. In t,he absence of an orphan sets are uniquely identified in the system by a unique
elimination sc.heme, the implementor of such a. type is I S - i d , i.e., corresponding to Unix process-id.
404
2.2 Distributed thread of control tories are responsible only for creating objects2 of a spe-
cific class. The interface to a factory object has one
'Traditional threads [Cooper 881 will be referred to as method ( c r e a t e ( ) ) , which creates insta.nces of a partic-
physical threads in this paper as they have their own ular class in the same address space where the factory
physical stacks. Typically, a physical thread is uniquely is. The c r e a t e ( ) method can be thought of as a wrap-
identified within a process by a t h r e a d - i d . Distributed ping to the n e w 0 operator in C++. Once the program-
thread (dtlwead) represents the sequence (flow) of events mer has a reference to a factory object, the programmer
or invocations by an application (one or more physical can create objects of that specific class by invoking the
threa.ds) in a distributed system, i.e., the dthread notion c r e a t e ( ) method on t8hat factory independent of the
is a virtual thread of control and it does not own private factory location.
physical stacks, rather it uses potentially many of the
st,acks belonging to existing physical threads. 2.4 Persistent objects
Ilthreads are created upon invoking operations on ob-
Persistent objects are objects whose lifetime is indepen-
jects asynchronously, i.e., an application might have
dent of the lifetime of the thread which created them.
a tree of distributed threads. A distributed thread
On the other hand, non-persistent object's lifetime is
terminates when the method originally invoked asyn-
tied to the lifetime of the thread which created them, for
chronously terminates.
example, in traditional Unix/C++ environment objects
The not,ion of dthread is used to help in tracing events are deleted automatically when the process terminates.
and cleaning orphans later. Each dthread'has a unique
identifier in the syst,em (dthrea.d-id) assigned at. cre-
3 Problem statement
ation time in a decentralized manner using the tuple
<machine-id : IS-id : thread-id>'.
The current memory management model for C++ is
to reclaim memory for all objects in a given process
2.3 Factories (address space) upon process termination. This model
would not, apply for persistent objects as by definition a
Fact,ories are objects specialized in creating objects of a persistent object's lifetime is independent of the lifetime
spwific class, i.e., interface and implementation. They of the dthread which created it.
provide a uniform model for creating objects in a dis-
On the other hand, applying the current C++ mem-
tribukd environment. This model allows the program-
ory management model to non-persistent objects is a
mer to create objects t,he same way independently of
much more difficult problem and needs to be resolved.
where they are created (i.e., in the local address space
For example, consider a dthread that creates a remote
or in a different one).
non-persistent object and then terminates. This remote
In traditional C++, the operator new(> creates a.n object becomes an orphan and needs to be reclaimed.
jnstance of a class only in the same address space. Fac- Next section presents a lazy scheme where orphans are
In a Unix context, machine-id corresponds to a host-id, IS-id 2Notice that factories only create objects and do not delete or
corresponds to a process-id, and thread-id is a unique ident.ifier manage them. Deleting an object will be dealt with in this paper
to a physical thread within that process. in a later section.
405
guaranteed t o be eliiilinated ai the termination of the cal &rectory interface declaration. Referring to
dthread which created them. figure 3 , an entry in the local directory object’s
database in IS2 is described below:
o b j e c t r e f e r e n c e > . Figure 1 represents t,he lo- 0 GID: is the object (object ) global unique-id.
406
inteif acc loc,al-directory =
add-entry(dthread-id ID, IS-id IS, gid-t GID, ref -t LCL-OBJ-REF) ,
delete-entry(ref-t LCL-OBJ-REF) ,
delete-entry(gid-t GID),
delete-entry(dthread-id ID, IS-id IS);
interface remote-directory =
add-entry(dthread-id ID, gid-t GID, ref-t RMT-LD-REF, ref -t RHT-OBJ-REF) ,
delete-entry(ref-t RMT-OBJ-REF),
delete-entries(ref-t RMT-LD-REF),
check(dthread-id ID) ;
e R,eiriot,e LD reference: is a remote reference to to get to them to delete t8he111. Access to these two
the loml directory object, in the implement,;?- directory objects is transpa.rent to the application pro-
tion set, where objectl is (i.e., LU2). granimer (i.e., the application need not t o do explicit
e Re1not.e object reference: is a. remote reference operations on these directory objects) and they are only
to t8heact,ual object (i.e., objectl) in 1’32. accessed from system funct,ions. The following a.re three
scenarios illustrat<ingwhen these directory objects are
The add-entry() method insert#sa tuple in t,he re-
a.ccessed:
mote directory’s st,ate about, a.n ol,ject, creat,ed re-
iiiot,ely from this iniplemeii tation set,.
o On creating a remote object, the factory’s
The delete-entry() inet.hod delet,es a reiiiot,e oh- create() method initializes an entry in the Iocal
ject and cleans the appropriate direct,ory entries. directory object, for the implementation set where
The delete-entries() method deletes all entries the factCoryis. Back on the client side, the c r e a t e 0
i n t,liis remote direct,ory object, for objects created method will initialize an entry in the remote di-
from this implenientlation set in a specific remote rectory object, before returning a reference to the
implenientmationset. applicatioii.
408
l o c a l d i r e c t o r y o b j e c t (LDI)
l o c a l d i r e c t o r y object(LD2)
I remote d i r e c t o r y
o b j e c t (RDl) 1 I \
\
I I
dthr I I 4
'
remote d i r e c t o r y
!
I
0: I
obj e c t (RD2 )
: 0: I
---*a
0
I
._----
I
I
I
I
I I
I
I
I
I
I
J
I
I
I
I I I / I
' I I I J
I / I / I
I I I / I
I f I / I
. n a tE
\ completed o r p h a n s
elimination
IS1 IS2
409
terminat,e(dthread-id ID)
begin
ref-t RMT-LD-REF;
while(RMT-.LD-REF : = remote-directory->check(ID))(
I1MT_LL)_REF->delete_entry(ID, IS);
remote_directory->delete-entries(RMT-LD-REF);
3
end ;
sel,s in a distributed system. tory object in tjhis implerneritatiori set (i.e., IS2). The
net result is deleting the object (objectl) and cleaning
5.3 Explicit object deletion the appropriate entry i n the local directory object (i.e.,
IS2).
Factories are used to provide a uniform method of crest,-
ing objects in a distributed system. They are responsi-
6 Conclusion
ble for creat,ing objects and iiot, for managing or deleting
them. This is because we believe that the programnier
The orphan detection and elimination issue has been
should not have to remember the factory reference in
investigated extensively in the distributed transaction
order to be able to delete an object. Iiistead, we intro-
contest as well as in the distributed system cont,ext
duce a new kill() function to replace the delete()
within t,he R P C framework. In this paper we ex-
function in C++.
tended this investigation to a general distributed object-
If the object t o be deleted is remote (i.e., oriented application environment. Orphans in a dis-
d t h r m d l in IS1 is deleting o b j e d l in 1S2, the tributed object-oriented application environment are
kill(obj2ef) function (see figures 3, 5) invokes the objects which are created by the application and con-
delete-entry(obj2ef) method 011 the remote direc- tinue to exist beyond the application’s lifetime. Orphan
tory (R,Dl) in this implementation set (El). This in objects in a distributed object-oriented application en-
turn invokes the delete-entry(G1D) method on the ap- vironment correspond t o activities which continue to ex-
propria.te remot,e LD object (i.e., LD2). The net result ecute on behalf of aborted transactions in a distributed
is deleting t,he remote object (objectl) and cleaning the transaction system.
appropriate entries in the directory objects, i.e., an en-
We presented a simple and a general model for de-
t8ryin the local directory object in IS2 and a n entry in
tecting arid eliminating orphans in a distributed object-
the remote directory object in IS1.
oriented application environment. Also, we presented a
On the other hand, if t,he object, is local (e.g., simple example to illustrate how our model would work
dthreadl in IS2 is deleting object 1 and obj -ref is a ref- in a distributed Unix/C++ application environment.
erence to objectl), the kill(objxef) function invokes An implenientsatlionof our model should not add much
the delete-entry(objief) method on the local direc- overhead; we are planning t o implement this model in
410
void* kill(ref,t obj-ref)
begin
i f (obj-ref->is-remote())
remote-directory->delete-entry(obj-ref) ;
else
local-directory->delete-entry ( o b j - r e f ) ;
end;
the near future as soon as the necessary platform envi- Systenas-an Advanced Course, R.
Ijayer, R..M. Graham, and G. Seeg-
ronment is completed. muller, Springer Verlag, New York,
pp. 393-48 1, ( 1978).
411
hility in Distrihted Sofi.ware a n d Updates,” J. ACM, vol. 26, No. 4.
Database Sysiems, pp. 42-, (Jaii- pp. 631-653, (Oct,ober 1979).
uary 1986).
[Ravindran 891 I<. Ravindran and S.T. Chanson,
[Mueller 831 E.T. Mueller, J.D. Moore, and “Failure Transparency in Remote
G.J. Popek, “A Nested Transaction Procedure Calls,” IEEE Trans.
Mechanism for LOCUS,” Proc. of Compt., vol. 38, No. 8, pp. 1173-
the Ninth AGM Symp. o n Operat- 1187, (August 1989).
ing Systems Principles, pp. 71-85,
(October 1983). [Shrivastava 831 S. Shrivastava, “On the Treatment
of Orphans in a Distributed Sys-
[Paiizieri 881 F. Panzieri and S. Shrivastava, tem,” Third Symp. 07). Reliability in
“Rajdoot: A Remote Procedure Distributed Soflutare and Database
Call mechanism Supporting Or- Sysfems, pp. 155-162, (Nov. 1983).
phan Detection and Killing,” I E E E
Trans. on Soft. Eng, vol. 14, No. I , [Walker 811 E.F. Walker, “Orphan detection in
pp. 30-37. (January 1988). t,he Argus system,” Massachusetts
Inst. technol., Lab. Comput. Sci.,
[Papadimitriou 791 C.H. Papadimitriou, “The Serializ- Tech. Rep. TR-326, (June 1984).
ability of Concurrent IIat,abasc
412