Eye of Ra
SECURITY RESEARCHAsbawy
cd ../writeups
2026-07-21·HackTheBox·Challenge·5 min

Curveware — HackTheBox Challenge Writeup

HardCryptography Windowsretired
CryptographyElliptic CurveHidden Number ProblemRansomwareLLLSageMathWindows
sharepostshare
_

Challenge Overview

Curveware is a hard Cryptography / Reverse Engineering challenge simulating a ransomware infection. We are given a 64-bit Windows executable (curveware.exe) and a directory of encrypted files appended with .vlny[10-hex].

Our objective is to reverse-engineer the cryptographic scheme, exploit partial nonce leakage via lattice reduction, and decrypt flag.txt.

_

Binary Analysis & File Structure

Static analysis with Ghidra reveals:

  • ▹Crypto Engine: Statically linked against libecc (ANSSI's Elliptic Curve library) using NIST P-256 (secp256r1).
  • ▹Key Generation: Uses CryptGenRandom to generate a single 32-byte secret key $d$, serving as both the AES-256-CBC key and the ECDSA private key.
  • ▹Nonce Leakage: The 10-character hex extension suffix (e.g., .vlny0742e9337a) exposes the lower 40 bits of the ephemeral nonce (k = SHA256(plaintext)).

::Encrypted File Layout

ComponentOffset / RangeSizeDescription
Signature $r$0x00 – 0x1F32 BytesECDSA signature component $r$
Signature $s$0x20 – 0x3F32 BytesECDSA signature component $s$
AES IV0x40 – 0x4F16 BytesAES-256-CBC Initialization Vector
AES Ciphertext0x50 – EOFVariableEncrypted file payload

To decrypt any file, strip the first 64 bytes (signature) and next 16 bytes (IV), then decrypt the payload using AES-256-CBC with the key $d$.

_

Cryptographic Vulnerability: Hidden Number Problem (HNP)

The binary computes signatures as:

~ / text
k ≡ r + s·d (mod n)

Since the lower 40 bits of k are exposed in the filename extension (leak), we express k as:

~ / text
k = leak + 2⁴⁰ · x (where x < 2²¹⁶)

::Deriving the HNP Relation

Substituting k into the signature equation:

  1. ▹

    Substitute k:

    ~ / text
    leak + 2⁴⁰ · x ≡ r + s·d (mod n)
  2. ▹

    Rearrange & multiply by (2⁴⁰)⁻¹ (mod n):

    ~ / text
    x - (2⁴⁰)⁻¹ · s·d + (2⁴⁰)⁻¹ · (leak - r) ≡ 0 (mod n)

With 18 encrypted sample files, we construct an $(m+2) \times (m+2)$ lattice matrix and apply LLL reduction to recover the secret key $d$.

_

Exploitation & Key Recovery

::1. Lattice Reduction (solve.sage)

~ / bash
sudo apt update && sudo apt install sagemath -y

Note: SageMath is not available on Ubuntu 22.04 LTS. You can install it by adding the EPEL repository to your APT sources.

~ / python
from pathlib import Path
from Crypto.Cipher import AES
from Crypto.Util.Padding import unpad
from Crypto.Util.number import bytes_to_long, long_to_bytes
 
n = 0xffffffff00000000ffffffffffffffffbce6faada7179e84f3b9cac2fc632551
B = 2^40
Binv = pow(B, -1, n)
 
root = Path('business-ctf-2025-dev')
files = sorted(root.rglob('*.vlny*'))
 
r, s, leak = [], [], []
 
for f in files:
blob = f.read_bytes()
r.append(bytes_to_long(blob[ :32]))
s.append(bytes_to_long(blob[32:64]))
leak.append(int(f.suffix[5:], 16))
 
m = len(r)
a = []
t = []
 
for ri, si, li in zip(r, s, leak):
t.append((Binv * si) % n)
a.append((-Binv * (li - ri)) % n)
 
M = identity_matrix(QQ, m)*n
M = M.stack(vector(t))
M = M.stack(vector(a))
M = M.augment(vector([0]*m + [B/n, 0]))
M = M.augment(vector([0]*m + [0, B]))
 
for row in M.LLL():
if abs(row[-1]) == B:
d = row[-2]*n/B % n
print(f"d = {hex(d)}")
break
 
key = long_to_bytes(d)
~ / bash
sage solve.sage

Recovered Private Key: 0xc5120eda0305ce74a125b5bd727e4fee5a24457ab376b69578c179f8440881e0

::2. Decryption & Flag Capture (solve.py)

~ / python
#!/usr/bin/env python3
import sys
import argparse
from pathlib import Path
from Crypto.Cipher import AES
from Crypto.Util.Padding import unpad
 
def solve(target: str = None, port: int = None) -> str:
key = bytes.fromhex('c5120eda0305ce74a125b5bd727e4fee5a24457ab376b69578c179f8440881e0')
flag_file = Path('business-ctf-2025-dev/crypto/curveware/flag.txt.vlny0742e9337a')
 
blob = flag_file.read_bytes()
payload = blob[64:]
iv = payload[-16:]
ct = payload[:-16]
 
aes = AES.new(key, AES.MODE_CBC, iv)
pt = unpad(aes.decrypt(ct), 16)
return pt.decode().strip()
 
def main():
print(f"[+] FLAG: {solve()}")
 
if __name__ == "__main__":
main()
HTB Challenge Flag
••••••••••••••••••••••••••••••••••••••••••••••••[ Click to reveal flag ]
_

References

  1. ▹Hidden Number Problem (HNP): D. Boneh and R. Venkatesan, "Hardness of Computing the Most Significant Bits of Secret Keys in Diffie-Hellman and Related Schemes", CRYPTO '96.
  2. ▹ECDSA Nonce Attacks: N. Smart, "The Exact Security of ECDSA of Small Nonces", 2001.
  3. ▹libecc Library: ANSSI Open Source Elliptic Curve Cryptography Library — GitHub Repository.
  4. ▹SageMath LLL Reduction: SageMath Documentation on Integer Lattices and LLL Reduction.
sharepostshare
▸about the author
Eye of Ra
Asbawy(Mohammed Al-Kasabi)

Red Team Consultant · Penetration Tester · Bug Bounty Hunter

Offensive security professional with 250+ vulnerabilities reported across 50+ organizations including Atlassian, Vimeo, and AT&T. Sharing research, tools, and field notes.

githubmediumlinkedin
// end of writeup — return /writeups