Flexlm: Recover LM_SEEDS based on encryption seeds

[dirkmill]

hi tr1stan,

i'm afraid your efforts regarding cro/trl are pointless.
if you have a look at the following code you will understand the problem:

l_genrand(job, lmseed1, lmseed2, lmseed3, NEWSEEDSIZ, newseeds);
for (i = 0;i < 4; i++){
   seed1 |= newseeds[i] << (i * 8);
   seed2 |= newseeds[i+4] << (i * 8);
   seed3 |= newseeds[i+8] << (i * 8);
   seed4 |= newseeds[i+12] << (i * 8);
   croseeds[0][0] |= (newseeds[i+16] << (i * 8));
   croseeds[0][1] |= (newseeds[i+20] << (i * 8));
   croseeds[0][2] |= (newseeds[i+24] << (i * 8));
};

the only knowledge accessible in a production build of an isv's flexlm binaries are seed1 and seed2 (usually called encseed[0]/[1] in the tuts) and some partial information about seed3/4 recoverable from lm_new.
l_genrand is based on certicom security-builder routines for the generation of random "newseeds" based on lmseed1/2/3 as prng-seeds. this function is costly and one-way.

also your estimation of the size of the keyspace in case of a birthday attack seems to be off by a huge amount. the size of returned data as can be seen above is way larger (28 bytes at least) than you expected, so a birthday attack is unlikely to be cheaper than a full brute-forcing of the input-space namely lmseed1/2/3.

sorry to crush your hopes

dirkmill