A Roblox module that compresses various data types into a buffer for easy storage, compression and encryption.
DEFLATE/Zlib module not created by me, see the original asset here
Either download the rbxm from the Releases section, or set it up yourself with the 4 source scripts.
NetShrink is the main script you will be requiring. If you are setting up manually,
the three other scripts should be children of the NetShrink module as demonstrated.
To encode data into a buffer, you call the NetShrink.Encode function which takes a variable number of arguments.
If you want to encode a table of arguments to avoid register limits, NetShrink offers a variant called NetShrink.EncodeT
These arguments you send will be the variables you compress into the buffer for transmission.
Here is a code example of how you encode data:
local encoded = NetShrink.Encode(
123,
{["test1"] = "test2"},
0.5
)
print("Successfully encoded to "..buffer.len(encoded).." bytes.")
To reduce data usage, see the section "Optimizing data usage"
To optimize performance, see the section "Optimizing performance"
Once you've ran NetShrink.Encode and gotten your buffer, you can also choose to encrypt it using NetShrink.Encrypt
This function takes two arguments, the buffer and a numeric key to use for encryption and it will return the encrypted buffer.
The encryption works by using the number as a seed to randomly XOR shift every single byte.
To decrypt you have two options, either use NetShrink.Encrypt again with the same key, or see the section "Decoding data"
To decode data from a buffer, call the NetShrink.Decode function.
This function takes a buffer as an input but also optionally a boolean and a number.
The buffer is of course what's being decoded but if you send true as the second argument,
the function returns the decoded variables in a table instead of multiple return values.
If a third argument is given (must be number) then it will decrypt the input buffer with the argument as the key, before decoding.
This argument must be used if you are decoding an encrypted buffer and the key must match what was used during encoding.
If we are trying to decode our example transmission, here's a simple example:
print(NetShrink.Decode(encoded,true)) -- prints a table
print(NetShrink.Decode(encoded)) -- prints: 123 {...} 0.5
If encoded was encrypted, adding the key used during encoding as the third argument to Decode will make sure the buffer is read correctly.
Compression is an expensive part of NetShrink.
If you need to perform a lot of encode/decode operations for cases like Multiplayer, it is recommended to disable compression.
To do this, you mainly want to set NetShrink.Config.CompressMode to 0 to not pass the output through EncodingService.
This increases the output size, so check the section "Optimizing data usage" to find ways to counter that.
Now that NetShrink's recommended encoding method is to handle type conversion automatically,
there are some configs offered to control how aggressive the compression should be for auto conversion.
These settings are accessible through NetShrink.Config.AutoConversion and here are all the currently available settings:
Controls the compression method that is attempted on all converted strings. Unnecessary and not recommended for performance.
Default value: 0 (None)
Controls the compression level that is used with the compression method, ignored if CompressMode is 0.
Default value: 1
Compresses all floating point numbers as 32-bit, not 64-bit, cutting data size and precision in half. Applies to:
- Decimal Numbers
- Vector2/Vector3
- CFrame
- Color3 (if Use3bColors is false)
Default value: false
Compresses every Color3 channel as a UInt8 instead of a floating point number, reducing size from 12/24 bytes to 3 bytes.
Default value: true
Compresses CFrames with only XYZ coordinates and euler angles.
Do not enable, compressed size is worse on v1.5.2, improvements coming soon.
Default value: false
This setting influences detection between tables and dictionaries, if a table like {true,nil,false} is passed,
it notices an index jump from 1 to 3 but encodes the missing index as nil if the table is not detected as a dictionary.
Default value: true
This setting only matters if IncludeIndexHoles is true, this limits the max allowed index jump from nil values when checking
if a table is a dictionary for safety reasons. If you have way too many nil values separating non-nil values in a table try increasing this value to preserve them.
Default value: 10
There are also settings available for how the entire buffer gets compressed, accessible in NetShrink.Config:
The compression method that will be used on the final buffer output to reduce size. Not recommended for performance.
Default value: 3 (EncodingService Zstd)
Supported values: 0: None, 1: DEFLATE, 2: Zlib, 3: EncodingService Zstd (EncodingService should be faster as it is native)
The compression level that will be used by the compression method, ignored if CompressMode is 0.
Default value: 1
Adds debug profiling for encode/decode processes to measure execution time in the Micro Profiler.
Default value: false
Before NetShrink updated to v1.3, you would have to convert your variables to NetShrink data types manually.
This is handled automatically now, but you also have the choice to do the conversion yourself with NetShrink.EncodeManual
Here's a code example of encoding with EncodeManual, and below you will find Documentation of all types you can encode.
local encoded = NetShrink.EncodeManual(
NetShrink.UInt8(127),
NetShrink.UInt16(65533),
NetShrink.UInt32(4294967295),
NetShrink.Table(NetShrink.Single(0.5)),
NetShrink.Dictionary({[NetShrink.String("test",0,0)] = NetShrink.Boolean5(true)})
)
Below is a list of all supported data types and their respective functions and documentation.
- String
- Boolean5
- UInt8
- UInt16
- UInt32
- Single
- Double
- Vector2
- Vector2int16
- Vector3
- Vector3int16
- CFrame
- CFrameEuler
- Color3
- Color3b
- ColorSequence
- Table
- Dictionary
- Nil
- EnumItem
- UDim
- UDim2
- NumberSequence
- NumberRange
Stores a string with optional compression methods.
Arguments: input: string, compressMode: number, compressLevel: number
compressMode: Controls what compression method to use, (0: None, 1: DEFLATE, 2: Zlib, 3: EncodingService Zstd)
compressLevel: Controls the compression level, higher takes longer to process, range: 0-9
Example: NetShrink.String("aaaaaaaaaaaaa",1,9)
Stores up to 5 booleans into one byte.
Arguments: ..., only booleans can be sent, exceeding 5 arguments or sending none causes an error.
If more than one boolean is encoded, it decodes as a table of booleans.
Example: NetShrink.Boolean5(true,true,false,false,true)
Stores a number from 0-255 into one byte.
Arguments: num: number, any number out of range will cause an error
Example: NetShrink.UInt8(127)
Stores a number from 0-65535 into one byte.
Arguments: num: number, any number out of range will cause an error
Example: NetShrink.UInt16(32767)
Stores a number from 0-4294967295 into one byte.
Arguments: num: number, any number out of range will cause an error
Example: NetShrink.UInt32(2147483647)
Stores a number as a 4-byte single-precision floating point. This risks losing some precision over normal number variables.
Arguments: num: number
Example: NetShrink.Single(34578547893347589) (this loses precision and becomes 34578547624378370)
Stores a number as a 8-byte double-precision floating point. The standard number variable data type.
Arguments: num: number
Example: NetShrink.Double(34578547893347589)
Stores a Vector2 with an option to use single-precision to reduce size by half.
Sizes: Single-precision: 8 bytes, Double-precision: 16 bytes.
Arguments: input: Vector2, float: boolean, setting float to true will encode the Vector2 as single-precision, sacrificing precision for size.
Example: NetShrink.Vector2(Vector2.new(384956,29538),true), this encodes as single-precision.
Stores a Vector2int16 into 4 bytes.
Arguments: input: Vector2int16
Example: NetShrink.Vector2int16(Vector2int16.new(32767,-32768))
Stores a Vector3 with an option to use single-precision to reduce size by half.
Sizes: Single-precision: 12 bytes, Double-precision: 24 bytes.
Arguments: input: Vector3, float: boolean, setting float to true will encode the Vector3 as single-precision, sacrificing precision for size.
Example: NetShrink.Vector3(Vector3.new(384956,29538,347835),true), this encodes as single-precision.
Stores a Vector3int16 into 6 bytes.
Arguments: input: Vector3int16
Example: NetShrink.Vector3int16(Vector3int16.new(32767,-32768,16384))
Stores a CFrame into 24 bytes.
Arguments: input: CFrame
Example: NetShrink.CFrame(workspace.SpawnLocation.CFrame)
Stores a CFrame with an option to use single-precision to reduce size by half.
This variant only stores XYZ coordinates and XYZ EulerAngles from the ToEulerAnglesXYZ function to save space.
Sizes: Single-precision: 24 bytes, Double-precision: 48 bytes.
Arguments: input: CFrame, float: boolean, setting float to true will encode the CFrame as single-precision, sacrificing precision for size.
Example: NetShrink.CFrameEuler(workspace.SpawnLocation.CFrame,true), this encodes as single-precision.
Stores a Color3/BrickColor with an option to use single-precision to reduce size by half.
Sizes: Single-precision: 14 bytes, Double-precision: 26 bytes.
Arguments: input: Color3/BrickColor, float: boolean, setting float to true will encode the color as single-precision, sacrificing precision for size.
Example: NetShrink.Color3(Color3.fromRGB(255,127,64),true), this encodes a Color3 as single-precision.
Example #2: NetShrink.Color3(BrickColor.new("Bright red")), this encodes a BrickColor as double-precision.
Stores a Color3/BrickColor as a 3-byte RGB value from 0-255. Any number outside this range will be clamped.
Arguments: input: Color3/BrickColor
Example: NetShrink.Color3b(Color3.fromRGB(255,127,64)) or NetShrink.Color3b(BrickColor.new("Bright red"))
Stores a ColorSequence with two options, "float" for encoding all decimal numbers as single-precision and "byte" for using 3-byte colors.
Sizes: 3 bytes + 7/11/16/32 bytes, size varies with input settings.
Arguments: input: ColorSequence, float: boolean, byte: boolean
Example: NetShrink.ColorSequence(ColorSequence.new({ColorSequenceKeypoint.new(0,Color3.fromRGB(255,0,0)),ColorSequenceKeypoint.new(1,Color3.fromRGB(0,0,255))}), true, true)
Accepts a variable number of data type arguments and instructs NetShrink to encode them into a table.
Tables can be placed within eachother endlessly. Cost per table is 1.25 bytes.
Arguments: ...
Example: NetShrink.Table(NetShrink.UInt8(127),NetShrink.UInt16(32767))
Accepts a table with NetShrink DataType keys & values and encodes as a dictionary.
Like with tables, you can have dictionaries in dictionaries. Cost per dictionary is 1.875 bytes.
Arguments: input: {}
Example: NetShrink.Dictionary({[NetShrink.String("testKey",0,0)] = NetShrink.UInt8(123)})
Stores a nil value, that's about it.
Example: NetShrink.Nil()
Stores an EnumItem as one 1-byte value and one 2-byte value.
Arguments: input: EnumItem
Example: Netshrink.EnumItem(Enum.EasingDirection.Out)
Stores a UDim with Scale and Offset as 8 bytes.
UDims are hardcoded as single-precision so double-precision is not available for this DataType.
Arguments: input: UDim
Example: NetShrink.UDim(UDim.new(120, 346))
Stores a UDim2 with Scale and Offset as 16 bytes.
UDim2s are hardcoded as single-precision so double-precision is not available for this DataType.
Arguments: input: UDim2
Example: NetShrink.UDim2(UDim2.new(120, 346, 81, 299))
Stores a NumberSequence with an option to use single-precision to reduce size by half.
Size: 2+(keypoints * 4) bytes as single-precision, 2+(keypoints * 8) bytes as double-precision.
Arguments: input: NumberSequence, float: boolean
Example: NetShrink.NumberSequence(NumberSequence.new(0, 10), true)
Stores a NumberRange with an option to use single-precision to reduce size by half.
Size: 8 bytes as single-precision, 16 bytes as double-precision.
Arguments: input: NumberRange, float: boolean
Example: NetShrink.NumberRange(NumberRange.new(3, 5), true)