MATH36206 Advanced Cryptology Week 2

2 Factoring
The security of many cryptosystems is based on the integer factorization problem; given a compos-
ite integer N , find any factor (p or q) such that N = pq.

For example we know that 33, 24, and 1309 can all be factored because 33 = 3 × 11, 24 = 2 × 12,
and 1309 = 7 × 187. The integers 13, 101, and 1049 cannot be factored because they are all primes.
So the integer factorization problem consists of determining if a given integer is in the following
set:
F ACT OR = {N |N = pq, for integers p, q > 1}
We have that 33, 24, and 1309 ∈ F ACT OR.

We have that 13, 101, and 1049 ∈

/ F ACT OR.

2.1 Factoring Algorithms

A factoring algorithm for a given integer N determines if N ∈ F ACT OR, and provides proof of
membership (this is also called a certificate) by returning any factor (p or q) of N .
√
2.2 Trial Division up to N method
It’s simple and effective (for small values of N ):

√ √
This algorithm takes a maximum of N steps to complete the for loop, so it’s O( N ). Al-
though this might seem fast it’s actually horrible when we evaluate the number of steps in terms
of the size of N in binary. For example, the input N = 221 is an 8-bit number. So the maximum
√
number of steps to complete the for loop is about 28 = 28/2 = 24 .

In general, if the√input N is an n-bit number, the maximum number of steps to complete the
for loop is about 2n = 2n/2 . So trial division is actually a O(2n ) (or exponential time) algorithm
where n is the size of the input in binary.

joshua.schneider@sheridancollege.ca page 1
MATH36206 Advanced Cryptology Week 2

2.3 Pollard p − 1 method

Let N = pq, and p − 1 = s1 s2 ...si be the prime factorization of p − 1.

Let B ∈ Z+ such that B! = B(B − 1)(B − 2)...(si )...(s2 )...(s1 )...(3)(2)(1) contains all factors
of p − 1 or q − 1 (but not both!).

Then (p − 1)|B!.

Compute a = 2B! mod N . We have that:

a = 2B! mod N
B!
a≡2 mod p
B!
(p−1) p−1
=2 mod p
B!
= (1) p−1 mod p
=1 mod p

Then a − 1 ≡ 0 mod p, and we have that p|(a − 1) and that p|N , so therefore p = GCD(N, a − 1).

2.4 Pollard p − 1 example

Factor N = 407 using B = 5.

a = 2B! mod N
= 25! mod 407
120
=2 mod 407
= 264 232 216 28 mod 407
= (49)(81)(9)(256) mod 407
= 100

p = GCD(407, 100 − 1) = GCD(407, 99):

407 mod 99 = 11
99 mod 11 = 0

p = GCD(407, 99) = 11 and N = 407 = 11 × 37.

Note: using B = 6 would not work as 6! = 720 contains the factors of both 11 − 1 and 37 − 1:

a = 2B! mod N
6!
=2 mod 407
720
=2 mod 407
512 128 64 16
=2 2 2 2 mod 407
= (367)(366)(49)(9) mod 407
=1

joshua.schneider@sheridancollege.ca page 2
MATH36206 Advanced Cryptology Week 2

Exercises
2.1 Factor N = 77 using the Pollard p − 1 method with:

a) B = 3
b) B = 4
c) B = 5

Show the computation of a and GCD(N, a − 1).

2.2 Factor N = 299 using the Pollard p − 1 method with B = 4, and determine the largest value
of B that can be used.

2.3 Factor N = 517 using the Pollard p − 1 method with B = 5, and determine the largest value
of B that can be used.

2.4 Determine the range for B that can be used when factoring N using the Pollard p−1 method.

a) N = 899 = 29 × 31
b) N = 11413 = 101 × 113
c) N = 8881 = 83 × 107
B
where pn is the nth prime number.
Q
2.5 Let #B be the primorial of B. #B = pn
n=1
3
Q
For example #3 = pn = 2 × 3 × 5 = 30.
n=1

a) Factor N = 407 using B = 3 and a modified p − 1 method where a = 2#B mod N .

b) Factor N = 4853 using B = 4 and a modified p − 1 method where a = 2#B mod N .

joshua.schneider@sheridancollege.ca page 3
MATH36206 Advanced Cryptology Week 2

2.5 Lenstra EC method

Recall that for an integer a to have an inverse a−1 mod N , a must be relatively prime to N . In
other words, GCD(N, a) = 1. Lenstra’s method for factoring N = pq looks for integers that do not
have an inverse a−1 mod N , and therefore are not relatively prime to N . If one of these integers
is found we have that GCD(N, a) = p or q.

2.6 Mathematical “Dark Arts”

Lenstra’s method searches for these “integers without inverse” on randomly generated elliptic curves
over ZN . Recall that these mysterious (and magical?) mathematical objects are described by:

y 2 = x3 + ax + b mod N

−1 mod N
(y2 − y1 )(x2 − x1 )
 P ̸= Q
λ=

(3x1 + a)(2y1 )−1 mod N
 2
P =Q
2
x3 = λ − x2 − x1 mod N
y3 = −λ(x3 − x1 ) − y1 mod N

Here are the main steps in the method:

1. Randomly select parameters a, x0 , y0 such that:

2≤a≤N −1
2 ≤ x0 ≤ N − 1
2 ≤ y0 ≤ N − 1

2. Compute b = y02 − x30 − ax0 mod N . This establishes a random starting point P = (x0 , y0 ) ∈
E, a random elliptic curve described by y 2 = x3 + ax + b mod N .

3. Compute (B!)P . If at any point during this computation the slope, λ, requires an inverse
that does not exist, then return the factor GCD(N, 2y1 ) = p or GCD(N, x2 − x1 ) = p.

4. If the computation of (B!)P is completed without finding a factor, return to step 1 and search
for the factor on another random elliptic curve.

2.7 The power of the dark side

Step 3. mirrors the Pollard p − 1 method where we computed a = 2B! mod N to obtain a factor
GCD(N, a − 1) = p. A limitation of the Pollard p − 1 method is that if a factor is not found with a
small value of B, the only choice we have is to increase the size of B. Lenstra’s method has no such
limitation; small values of B can be used, again and again, over different random elliptic curves
which tend to have different numbers of total points.

We can also modify Step 3. in Lenstra’s method by making substitutions for B! such as the
primorial #B for further efficiencies.

Lenstra’s EC method is currently one of the top three integer factorization algorithms (the other
two being the quadratic sieve and the number field sieve).

joshua.schneider@sheridancollege.ca page 4
MATH36206 Advanced Cryptology Week 2

Exercises
2.6 Factor N = 15 using the Lenstra EC method by computing 2P on the curve
y 2 = x3 + 2x + 9 mod N where P = (5, 12).

2.7 Factor N = 91 using the Lenstra EC method by computing 2P on the curve

y 2 = x3 + 14x + 40 mod N where P = (20, 65).

2.8 Factor N = 517 using the Lenstra EC method by computing 3P = 2P + P on the curve
y 2 = x3 + 281x + 166 mod N where P = (383, 489). Include the manual calculation of the
EEA for any inverses required.

2.9 Factor N = 221 using the Lenstra EC method by computing 5P = 2P + 2P + P on the curve
y 2 = x3 + 191x + 218 mod N where P = (87, 144). Include the manual calculation of the
EEA for any inverses required.

2.10 Determine the minimum number of additions required to compute the following for a point
P on an elliptic curve described by y 2 = x3 + ax + b mod p.

a) (7)P
b) (5!)P
c) (#4)P
d) (257)P

joshua.schneider@sheridancollege.ca page 5
MATH36206 Advanced Cryptology Week 2

Answers
2.1 a = 64, GCD(77, 63) = 7
a = 71, GCD(77, 70) = 7
a = 1, B is too large.

2.2 a = 27, GCD(299, 26) = 13

B ≤ 10

2.3 a = 408, GCD(517, 407) = 11

B ≤ 22

2.4 5 ≤ B ≤ 6
7≤B≤9
41 ≤ B ≤ 52

2.5 a = 122, GCD(407, 121) = 11

a = 1267, GCD(4853, 1266) = 211

2.6 (9)−1 mod 15 does not exist; GCD(15, 9) = 3

2.7 (39)−1 mod 91 does not exist; GCD(91, 39) = 13

2.8 EEA gives 517(-109)+461(120)=1, λ = 224, 2P = (295, 94)

(88)−1 mod 517 does not exist; GCD(517, 88) = 11

2.9 EEA gives 221(-10)+67(33)=1, λ = 35, 2P = (167, 150)

EEA gives 221(-5)+79(14)=1, λ = 60, 4P = (172, 213)
(136)−1 mod 221 does not exist; GCD(221, 136) = 17

2.10 4
9
10
9

joshua.schneider@sheridancollege.ca page 6