Figuring out table based encryption in assembly


I've been tracing an old (1996) utility that uses a very sophisticated
encryption algorithm for license info. It generates a table, uses
constantly changing keys, and does some really bizarre stuff with
multiplication, shifts, xor, and table lookups.

The crypto group doesn't seem to be very active or very helpful, so I
thought I'd ask here: is there any way to figure out if this is a
standard encryption method?

It's so complex, it seems unlikely some guy in his garage made it up.
But at the same time, I'm not sure how to tell if it is DES, Blowfish,
or something similar.

.