No-regret slice reservation algorithms

Jean-Baptiste Monteil, George Iosifidis, Luiz DaSilva

Trinity College Dublin, Delft University, Virginia Tech

IEEE ICC 2021

SATORI
Smart Networking in the Era of AI
Problem statement
Network Slicing markets are becoming important:
• Increasing softwarization of wireless networks1
• Proliferation of over-the-top service providers (SPs)2
Their efficacy is a challenging and open problem
• Different SPs have different requirements
• Operators have internal needs
• SPs need to reserve their slices
Design of robust online learning algorithms
• Use of online learning
• Use of Online Convex Optimization (OCO) theory

1 A. Garcia-Saavedra, X. Costa-Perez, D. J. Leith, and G. Iosifidis, “FluidRAN: Optimized vRAN/MEC

orchestration,” in IEEE INFOCOM 2018 - IEEE Conference on Computer Communications, 2018,
pp. 2366–2374.
2 K. Samdanis, X. Costa-Perez, and V. Sciancalepore, “From network sharing to multi-tenancy: The 5G
network slice broker,” IEEE Communications Magazine, vol. 54, no. 7, pp. 32–39, 2016.
1
Problem statement

Slices of

Slicing

network resources
Bidding

Service Providers Network Operator

• Virtualization of resources (vRAN, c-RAN, small cells, vBSs)

• Infrastructure as a Service (IaaS), in cloud computing
• Multiple SPs → competition for resources
• Bidding process (cloud marketplaces3 )
3 M. Khodak, L. Zheng, A. S. Lan, C. Joe-Wong, and M. Chiang, “Learning cloud dynamics to optimize spot
instance bidding strategies,” in IEEE INFOCOM 2018 - IEEE Conference on Computer Communications, 2018,
pp. 2762–2770.
2
Related works
Slice reservation papers4,5 :
• Multi-time scale approaches • Information on slice prices
• Nested optimization problems • Information/assumptions on the SP
demand
OCO literature6 :
• Reservation model for cloud resources • Known and fixed costs for the resources
• Adversarial distribution for the demand

4 H. Zhang and V. W. S. Wong, “A two-timescale approach for network slicing in c-ran,” IEEE Transactions
on Vehicular Technology, vol. 69, no. 6, pp. 6656–6669, 2020.
5 Y. Zhang, S. Bi, and Y. A. Zhang, “Joint spectrum reservation and on-demand request for mobile virtual
network operators,” IEEE Transactions on Communications, vol. 66, no. 7, pp. 2966–2977, 2018.
6 N. Liakopoulos, G. Paschos, and T. Spyropoulos, “No regret in cloud resources reservation with violation
guarantees,” in IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, 2019, pp. 1747–1755.
3
Research problem
At a high-level, the SP decides the slice size:
• Slice prices might be revealed only after the reservations (auction process)
• The SP demand evolves and is unknown

At a lower level, the SP decides the slice composition:

• Resource prices might be revealed only after the reservations (auction process)
• Contributions of each resource might evolve and are unknown to the SP
• The SP demand evolves and is unknown

Contributions:
• Design of online learning solutions for slice reservations
• Performance analysis of the proposed solutions
• Assessment of the efficacy of our AI solutions against different traces and scenarios

4
System model
Notations: ak the demand, pt the period price, qk the spot price, T the horizon, K slots per
period, B budget per period. The problem is:
T
X tK
X
(P) : max ak log(xt + yk + 1) (1)
{xt ,yt }T
t=1 t=1 k=(t−1)K+1
T
X
xt pt + yt> qt ≤ BT


s.t. (2)
t=1

Online convex form:

tK
X
(Pt ) : ft (xt , yt ) = − ak log(xt + yk + 1) → min (3)
xt ,yt
k=(t−1)K+1

gt (xt , yt ) = xt pt + yt> qt − B ≤0 (4)

5
Online Learning for Reservation (OLR) algorithm7

Compute
Reveal price Decisions Reveal and Update
and

(6) (7)
Next slot

We define the Lagrangian Lt (., .), z the reservation vector, λ the dual update:

kz − zt k2
Lt (z, λ) = ∇ft (zt )> (z − zt ) + λgt (z) + (5)
2ν

zt = arg min Lt−1 (z, λt ) (6) λt+1 = [λt + µ∇λ Lt (zt , λ)]+ (7)
z∈Z

7 T. Chen, Q. Ling, and G. B. Giannakis, “An online convex optimization approach to proactive network
resource allocation,” IEEE Transactions on Signal Processing, vol. 65, no. 24, 2017.
6
OLR – evaluation
OCO theory introduced by Zinkevich8 in 2003
Goal: no-regret slice reservation policy
The metrics:
P  
• Regret RT = Tt=1 ft (xt , yt )− ft (x∗t , yt∗ ) , where (x∗t , yt∗ ) is the optimal t-period
decision (i.e. the optimal solution of sub-problem (Pt ))
• Fit VT = Tt=1 gt (xt , yt )
P

Lemma 1
The performance analysis of OLR algorithm ensures that:

lim RT /T = 0, and lim VT /T = 0, given UgT = o(T ), and UzT = o(T )

T →∞ T →∞

8 M. Zinkevich, “Online Convex Programming and Generalized Infinitesimal Gradient Ascent,” in Proc. of
ICML, 2003.
7
OLR - system parameters

T
X
UzT = ||zt∗ − zt−1
∗
||, accumulated variations of the dynamic benchmark sequence, (8)
t=1

where zt∗ = (x∗t , yt∗ ) is the optimal solution of problem (Pt ).

T
X
UgT = max |gt (z) − gt−1 (z)|, accumulated variations of the constraints. (9)
z∈Z
t=1

• T time horizon (number of periods)

• K period length
• B budget per period
• D maximum reservation → ∀t, k, xt , yk ∈ [0, D] and zt ∈ Z = [0, D]K+1
• ν positive step-size of the primal update
• µ positive step-size of the dual update
8
OLR – results
We test our OLR algorithm under the stationary case:
• ak , qk and pt are uniformly distributed on [0, a], [0, q] and [0, p], respectively.

OLR
0.5

RT /T
0.0502

0.0

0.0

VT /T
0.0084

OLR
-0.5
0 250 500 750 1000
Time Horizon T

Evolution of RT /T and VT /T . Simulation parameters are set to T = 1000, a = p = q = 1,

D = 1.5 K = 3, UgT = 1000, B = 3, UzT = 762. Step-sizes: ν = 1, µ = 0.4.

9
OLR – results
We test our OLR algorithm under the stationary case:
• ak , qk and pt are uniformly distributed on [0, a], [0, q] and [0, p], respectively.

OLR

RT /T
2.0 0.0668

0.0
0.0

VT /T
−0.3037

-5.0
OLR

0 250 500 750 1000

Time Horizon T

Evolution of RT /T and VT /T . Simulation parameters are set to T = 1000, a = p = q = 1,

D = 20 K = 5, UgT = 20000, B = 10, UzT = 20282. Step-sizes: ν = 1, µ = 0.005.

10
OLR – results
We test our OLR algorithm under the stationary case:
• ak , qk and pt are uniformly distributed on [0, a], [0, q] and [0, p], respectively.

OLR

RT /T
2.0 −0.0022

0.0

0.0

VT /T
0.1001

-5.0
OLR

0 250 500 750 1000

Time Horizon T

Evolution of RT /T and VT /T . Simulation parameters are set to T = 1000, a = p = q = 1,

D = 20 K = 5, UgT = 20000, B = 10, UzT = 20282. Step-sizes: ν = 1, µ = 0.001.

11
OLR – results
We test our OLR algorithm under the non-stationary case:
• ak = sin(kπ/5) + unif[1, 5]
• qk = sin(kπ/5) + unif1, 2
• pt = sin(tπ/5) + unif[1, 2]

0.0
OLR

RT /T
0.057
-2.5

OLR
VT /T
2.0
0.059

0.0
0 250 500 750 1000
Time Horizon T

Evolution of RT /T and VT /T . Simulation parameters are set to T = 1000, D = 1 K = 3,

UgT = 982, B = 1, UzT = 673. Step-sizes: ν = 0.1, µ = 0.05.
12
OLR – Slice Orchestration
With the OLR-SO (SO for Slice Orchestration), we determine the slice composition, i.e. we
reserve a slice composed of different resources.
The vector θt measures the contribution of each resource on performance:
tK
X
ak log θt> xt + θt> yk + 1 ,


(Pt ) : ft (xt , {yk }) = − → min (10)
xt ,yk
k=(t−1)K+1
tK
X
gt (xt , {yk }t ) = x>
t pt + yk> qk − B ≤0 (11)
k=(t−1)K+1

xt and yk are vectors of [0, D]m , with a system of m resource instances.

The Lagrangian function Lt (z, λ) stays the same, with z ∈ Z = [0, D]m(K+1) .

13
OLR – Mixed-Time-Scale
With the OLR-MTS or OLR-SO-MTS (MTS for Mixed-Time-Scale), we update the decisions
yk ’s or yk ’s at each slot within the period.
Thus, the slotted problem becomes:

(Pk ) : fk (yk ) = −ak log(xt + yk + 1), → min (12)

yk
gk (yk ) = yk qk − Bslot ≤0 (13)

where xt is now a parameter (already decided) and Bslot = (B − xt pt )/K is the remainder of
the allocated budget.
Or:

(Pk ) : fk (yk ) = −ak log θt> xt + θt> yk + 1 ,

→ min (14)
yk

gk (yk ) = yk> qk − Bslot ≤0 (15)

where xt is now a parameter (already decided) and Bslot = (B − x>

t pt )/K is the remainder of
the allocated budget.

14
OLR – Mixed-Time-Scale
The per-slot Lagrangian is defined as (in the SO case):

ky − yk k2
Lk (y, λ) = ∇fk (yk )> (y − yk ) + λgk (y) + (16)
2ν 0

yk = arg min Lk−1 (y, λk ) (17)

y∈[0,D]m
0
λk+1 = [λk + µ ∇λ Lk (yk , λ)]+ (18)
The procedure (in the SO case) at each slot:

• ak−1 , qk−1 are revealed

• we decide yk
• ak , qk are revealed
• we decide λk+1

15
Extensions – results
We test our OLR-SO and OLR-SO-MTS under the stationary case:
• ak , qk and pt are uniformly distributed on [0, a], [0, q] and [0, p], respectively.

5.0
SO-OLR

RT /T
SO-OLR-MTS

0.2010

0.0

0.0

VT /T
0.0087
SO-OLR
SO-OLR-MTS
-10.0
0 250 500 750 1000
Time Horizon T

Evolution of RT /T and VT /T . Simulation parameters are set to T = 1000, a = p = q = 1,

0 0
D = [10, 10, 10], m = 3, K = 5, B = 15. Step-sizes: ν = 1, µ = 0.001, ν = ν, µ = µ

16
Extensions – results
We test our OLR-SO and OLR-SO-MTS under the non-stationary case:
• ak = sin(kπ/5) + unif[1, 2]
• qk = sin(kπ/5) + unif[1, 2, m]
• pt = sin(tπ/5) + unif[1, 2, m]
• θt = sin(tπ/5) + unif[1, 2, m]

0.0

RT /T
-1.0 0.1511 SO-OLR
SO-OLR-MTS

SO-OLR
VT /T
0.2401
5.0 SO-OLR-MTS

0.0
0 250 500 750 1000
Time Horizon T

Evolution of RT /T and VT /T . Simulation parameters are set to T = 1000, D = [2, 2, 2], m = 3,

0 0
K = 3, B = 5. Step-sizes: ν = 0.1, µ = 0.003, ν = ν, µ = µ 17
Next step – step sizes
So far, we have regret and fit bounds for the OLR and OLR-SO algorithms:

RT ≤ RTw (19)
VT ≤ VTw (20)

We define ∆α = RTw + αVTw .

We aim to minimize ∆α (ν, µ) wrt ν and µ.
This requires knowledge or estimates of UgT and UzT .
Having periodic returns of the regret/fit values would enable to adapt the step-sizes:
• low/negative regret, high fit → decrease ν or increase µ (large µ favorites lower
reservations)
• high regret, negative fit → increase ν or decrease µ (large ν favorites larger reservations)

18
Publications
• J. Monteil, J. Hribar, P. Barnard, Y. Li, and L. A. DaSilva, “Resource reservation within
sliced 5G networks: A cost-reduction strategy for service providers,” in 2020 IEEE
International Conference on Communications Workshops (ICC Workshops), 2020, pp. 1–6
• (accepted) J. Monteil, G. Iosifidis, and L. A. DaSilva, “No-regret slice reservation
algorithms,” in ICC 2021 - 2021 IEEE International Conference on Communications
(ICC), 2021
• finishing the journal extension

19
Questions?