Friday, January 18, 2013

Looking for JavaScript Decompressor for Data Compressed by C#

It happened that I needed to make an HTML page.  Sometimes such things happen, you know.  That page was required to be completely offline, just a snapshot of some piece of data being downloaded by users for further analysis.  At the same time that offline page was required to be as interactive as the online page with the same data.

The online page for that data contained grid with "master" rows and upon clicking on the row it presented a dialog box with "details" loaded from the server by AJAX request.  The offline page could not use AJAX so it should contain all needed data, both "master" part and all the "details", inside it.  The average amount of data used on these pages was not very small - about 3 megabytes in JSON format, so it was decided to compress that data on the server side and decompress needed pieces on the fly when they should come into the dialog.

The code for the server side was not very complex.  As the web application uses ASP.NET MVC and is written on C# I could use standard .NET classes for compression like either System.IO.Compression.DeflateStream or System.IO.Compression.GZipStream:


var details = new Dictionary<string, string>();
foreach (DataRow row in detailsData.Rows)
{
    var text = Convert.ToString(row["details_content"]);
    var bytes = Encoding.UTF8.GetBytes(text);
    using (var ms = new MemoryStream())
    {
        using (var zipStream = new DeflateStream(ms, CompressionMode.Compress))
        {
            zipStream.Write(bytes, 0, bytes.Length);
            zipStream.Flush();
        }
        details[Convert.ToString(row["master_id"])] = Convert.ToBase64String(ms.ToArray());
    }
}
viewModel.ZippedResultsJson = EncoderHelper.SerializeToJsonString(details);

But what to do on the page?  Are there any JavaScript implementations of, say, inflaters?  Initially I thought that these days there are lots of them as probably almost all things are already implemented in JS.  But when I tried to look for them I've found that things are not that good.  Actually there are just a few implementations of deflate/inflate algorithms and not all of them look working.  Thanks StackOverflow, they know answers to almost all our questions and I've found lots of answers how to get some of JS decompressors to work with Java or Python server-side compression.

But just a pair of questions about C#-JS were answered like "just use the browser gzip support", i.e. encode the whole response.  This is not my case as I don't have response so I tried to use some of implementations to see which of them is more usable.  Things were worse than I expected, no one of relatively small inflate implementations worked with C#'s DeflateStream-compressed data (although they worked quite fine with data compressed by themselves).

So I decided to try gzip instead of the "raw" deflate.  And I've found the only JS implementation of gzip decompressor on http://jsxgraph.uni-bayreuth.de/wp/2009/09/29/jsxcompressor-zlib-compressed-javascript-code/.  I tried it and it worked!  (Of course, I've changed the DeflateStream to the GZipStream on the server side).  Lots of thanks to those guys who created that really working tool!  With it all I needed on the page was just:

$('#details-div').html(JXG.decompress(detailsData[masterId]));

And that's it!  Simple, usable, great!  And it does Base-64 decoding inside so there is no need to use separate decoding tools or libraries.  Great job guys!  Now I know the right tool for this task.

X-Posted to: http://hill30.blogspot.com/2013/01/looking-for-javascript-decompressor-for.html

4 comments:

  1. This looks to be exactly what I'm looking for, many thanks.

    Problem is I tried to use JXGCompressor but get this problem right at the beginning

    http://www.screencast.com/t/wRjGbIxDUk

    Have you come across this in the past?
    Hope you can help
    Thanks!

    ReplyDelete
    Replies
    1. Thanks for your comment. Actually I've read a lot of articles and posts while trying to find out the working solution. And in some of them I saw this kind of exception. As far as I understand it can be caused by non-matching algorithms on the server and client sides. For example I did not find anything working with data compressed with System.IO.Compression.DeflateStream. The only working pair I've found is System.IO.Compression.GZipStream on the server side and JSX (de)compressor on the client side.

      Delete
  2. Thanks for your response Dmitry, that helped me to find out that the default gzip implementation in my .net 3.5 was the cause of the problem. I used ionic's zlib to produce a compatible string for JXG in the end.

    Hope this helps someone with a similar issue
    http://stackoverflow.com/questions/14993029/jsxcompressor-usage

    ReplyDelete
    Replies
    1. Wow, good to know that! Actually I use .NET 4.0 and it looks like this version has its default gzip implementation corrected... Thanks!

      Delete