## 0x00 Happy!

We are given an archive of challenge files: a Ruby script implementing RSA, a marshalled public key and an encrypted flag. The task, of course, is to decrypt the flag.

Auditing the script reveals that besides the usual parameters $$\{p,q,e,N\}$$, it stores a bunch of derived values $$\{d_1, d_2, c_f\}$$ in the Key object. These are later used during decryption. There is also something strange going on – keygen takes an extra argument $$k$$, which is the power to which $$q$$ will be raised in the public modulus:

Generally speaking, this is a bad idea. While factoring a product of two primes of roughly the same size (e.g. 512 bits) is infeasible, Boneh et al. show that factoring a number of the form $$pq^k$$ becomes significantly easier as $$k$$ increases. But let’s keep looking, maybe there are more issues.

While the padding scheme used looks legit1, there is a typo in the sanitization from private to public key:

The primes and some of the derived values are stripped, but $$c_f$$ sneaks through. Loading the public key file confirms this and we have three values to work with:

At this point in the CTF, we noted that the size of $$N$$ is $$\log_2{N}\approx 2295=765*3$$. Knowing that $$N=pq^k$$ and that both primes are roughly the same size (but at least 700 bits), we can conclude that $$k=2$$. This is sadly too small to enable direct factorization (ch. 6 in Boneh et al.), but maybe we can somehow use the information from $$c_f$$. The expression by which it is given can be simplified using Euler’s theorem:

\begin{aligned} c_f & \equiv p^{q^{k-1}(q-1)-1} \equiv p^{\varphi(q^k)-1} \equiv p^{-1}\ (mod\ q^k) \\ & \equiv p^{-1}\ (mod\ q^2) \end{aligned}

What we have, then, is simply the inverse of one factor of $$N$$ modulo the other two. This reveals the meaning behind the challenge description: “No, we’Re not SAd. We are Happy!”. It’s a reference to “R U Sad”, a challenge from this year’s Plaid CTF. In that one, we were given both inverses and could solve it by calculational reasoning with Bézout’s identity. Here, the situation is not so clear, which is reason enough to be sad.

## 0x01 Lenstra, Lenstra, Lovász

As a wise man once said, “If it’s not LLL, it’s LLL”. LLL is a lattice basis reduction algorithm which, given a basis for an integer lattice, is able to find a small basis for the same lattice with a precise bound on the coefficients. The details of how it works are not terribly important as we can just use an existing implementation.

As originally realised by Coppersmith and then refined by Howgrave-Graham, this algorithm is extremely useful for cryptanalysis. A. May’s survey provides a comprehensive overview of the area and was indispensable in solving this challenge. It contains a proof of the following:

Theorem 1. Let $$N$$ be an integer of unknown factorization, which has a divisor $$b \geq N^\beta$$, $$0 \lt \beta \leq 1$$. Let $$f(x)$$ be a univariate monic polynomial of degree $$\delta$$. Then we can find all solutions $$x_0$$ for the equation $$f(x) \equiv 0\ (mod\ b)$$ with $$\lvert x_0 \rvert \leq cN^\frac{\beta^2}{\delta}$$ in time $$\mathcal{O}(c\delta^5\log^9N)$$.

In other words, we can find some roots of a polynomial modulo $$b$$ in sorta-logarithmic time as long as they are small enough – asymptotically below $$N^{\frac{\beta^2}{\delta}}$$. The survey also lays out Coppersmith’s algorithm which finds such solutions2 (see “Coppersmith’s method in the univariate case”).

While the algorithm itself has strikingly low time complexity, I found one thing in its statement even more surprising – the fact that at no point do we need to know the value of $$b$$ to find the roots of $$f(x)$$ modulo $$b$$. It suffices to know that $$b$$ factors $$N$$ and that it’s at least $$N^\beta$$. To me, this reads like black magic. It also happens to be the case in our task, so let’s use it.

The crucial condition Coppersmith’s method needs to work are small-enough roots. We have the polynomial $$f(x)=c_fx-1$$, with $$f(p) \equiv 0\ (mod\ q^2)$$ and degree $$\delta = 1$$. To compute the smallness requirement, note that $$b = q^2 \approx N^\frac{2}{3}$$, so that $$\beta = \frac{2}{3}$$. Hence, we need $$\lvert p \rvert \leq cN^\frac{4}{9}$$, which is true, but not by much – $$p \approx N^\frac{1}{3}$$.

At this point it becomes clear why the modulus is so weird – this wouldn’t be possible to solve if $$N$$ were simply $$pq$$, because then $$\beta=\frac{1}{2}$$ and $$p$$ would be too large. Unless there is a completely different solution – I’d be interested in hearing about that!

One small issue remains - our polynomial is not monic. This can be solved by multiplying it by the inverse of $$c_f$$ modulo $$N$$ (remember, no need to know $$b$$) in the polynomial ring over integers modulo $$N$$.

Knowing all this, decrypting the flag amounts to typing the parameters into an off-the-shelf implementation of Coppersmith’s algorithm.

Thanks goes to my teammates hyperreality and Retr0id for helping out, as well as to the organizers for prompting me to finally learn LLL-based attacks.

1. And anyway a padding oracle wouldn’t be very useful without a remote service to connect to.↩︎

2. By which virtue it is a constructive proof of Theorem 1.↩︎