Solved – Test randomness of a generated password

On top of Daniel's great suggestion to use the information measures, you can consider breaking down the characters into groups to overcome the limitation of having to deal with way too many combinations. A natural breakdown is to capital letters (26), lower case letters (26), numbers (10), and other symbols (5-15 depending on implementation). So instead of having 70 independent symbols, you can deal with groups that have probabilities 26/70, 26/70, 10/70 and 8/70.

Alternatively, you can consider the transition probabilities from one character to the next. The password "blah273blah" would imply the transition probabilities "lower case -> lower case" of 6/10, "number -> number" 2/10, "lower case -> number" of 1/10, and "number -> lower case" of 1/10. All other probabilities are zeroes. These should be compared to the uniform transition probabilities (given above), although arguably Pearson $chi^2$ will hardly work well with so many zero cells. I guess this is an extension of the run test for binary events. I am sure a multinomial extension exists and is applicable to this situation.

In any case, you need to have a reference distribution of whatever "test statistic" you will end up with, which can be obtained by simulation from your own reliable source of random passwords. It is arguably better to use the true random numbers for this purpose, e.g., from http://www.random.org.