In-Memory data compression

[orlandod]

I am playing around with the idea of compressing cached data in order to speed up some of the processes that I run which receive multiple requests. I am storing the results in the workspace where I can then do a lookup. If the data is kept in-memory, it starts utilizing too much memory and becomes a drag on the system but it returns a response much quicker. Does anyone have any recommendations/functions for compressing/decompressing in-memory data (integers, doubles of vectors, arrays)?

Thank you,

Orlando

[orlandod]

Searching on Google led me to this article:

http://www.vector.org.uk/archive/v181/hack181.htm

Thanks to Stefano and Adrian

[StefanoLanzavecchia]

Hi! I am the Stefano who originally wrote the code to interface ZLIB with Dyalog APL and I hope you find it useful. But if you don't, you can also check http://www.dyalog.dk/dfnsdws/n_Data_compression.htm. Have fun :)

[AndyS|Dyalog]

Better would be http://www.dyalog.com/dfnsdws/n_Data_compression.htm .. one day we will disable links to dyalog.dk.

The "See also" link to pack on the above page may be of interest .. rather than attempting to compress data, pack attempts to reduce the amount of workspace used by sharing sub-arrays, although that is very much data dependent.