Normalization

[dmitrii-codes]

1. Two parameters were added:

• AzureML to use a web deployed model on Azure ML (API settings are required).
• Normalized to pick only normalized and important features. New vectors (weights) were obtained;

To normalize every feature, I used the data binning technique. It is a way to group
widely distributed (non-linear) features into a smaller number of "bins" (100 by default).
https://en.wikipedia.org/wiki/Data_binning
To get rid of unimportant features, I only picked those features where number of unique buckets is more than some specific number (I set 3 by default, in the future it is recommended to compare other values). The full report can be provided upon request.

2. settings.json is used to store the feature bins, indexes of the important features and API settings, as well as the weights and parameterized threshold;
3. Minimum version of the Windows PowerShell host was lowered to 1.8.3 so users could use VS Code debugger.
4. Fixed "empty script" bug -> empty script is not obfuscated a priori, no errors need to be thrown.

[edyoung]

These changes were implemented by Dmitrii as part of work to improve the accuracy of Revoke-Obfuscation on modules submitted to the PowerShell Gallery. We'd like to contribute them upstream. @LeeHolmes

[edyoung]

We probably want to tidy up the interface a bit. The names of the parameters are not very obvious, and it's possible to specify both -NormalizedFeatures and -WebScan, which I don't think makes sense. How about an optional -Model parameter, which can take enum values Standard, Normalized, or AzureML? Default would be Standard

[dmitrii-codes]

Yes, it is possible to specify both -NormalizedFeatures and -WebScan, but it will only run WebScan anyway.
I changed the names of the parameters to 'Normalized' and 'AzureML', but this interface is used in some functions that I haven't even looked at before. I can work on it when I'm done with my other tasks.

[edyoung]

Not sure why some funcs have Deep as an option and others don't. Does it still make sense to keep this as an option? In which case maybe it's another value which can be passed to -Model. If it doesn't outperform the other models maybe just remove it

[edyoung]

@danielbohannon can you give us an idea whether you'll be open to merging this kind of thing?