feat: added ability to receive a base64 with compressed clientstate, #6918 (#6919)

I encountered a problem with large URLs when using the "clientstate" API
with base64 strings: see #6918

* I found that it was quite easy to include this feature, just by
searching for `/clientstate/`.
* I "tested" my feature interactively with `npm run dev` and this
[link](http://localhost:10240/zclientstate/eNq1VduK2zAQ/ZXBhWITJc6lVye7sBT6uIWl9CUOiyJPYm1syUjyZsOSf+/Idm5tWuhD7YCs0cyZM5doXgOL1kqtbJDA/DWQGa0jBkHB1brma6RtIHq9gERW10Y0glTFMYiqMrhCg0pgArlzlU3iGNXg/GAgdBlvYxLFQivHpUITl5mtuErVG6lEUWcIM6mtM8jL23PhAdLw7WAtXV4va4vGw6ByDfBGbzbadnixlWpdYD9Hnh2dDPKqIlDFS6StQLAuwxe48es0Vf51WFYFdzgTBbcW7m6BgpNrpQ2Cy6WFNDiopAHkFBeDlTZ0hlDqkrik6lnLDCojlRtnITG0Du7e8ghegR6Cu/KEN1ztwO0qhLsoVV5EqCFBgCR+wyktM+ADfPHhhsNoCr2eJMhW91z/qdV/OtcfNfpPF/r+obiTROjawYy0Q8lIhb4osjSYeqpfer3xJAE+p6PFyXZ/+ryASIOUkhhM22PS2v8WKYHeawc211uVwE7XILgCX0ejC9jmUuSQyRIo1dmOKiXFJeUjClfZb+ZNFbDUZgcF33lS4ddvD98f7u7jL5RWn56Sui48JsJLMu74fHHzOmJjNmHv2Hv2gX1kn9jnfRcHr50G3rYJviRJ20yhtyODSdRkKke5zh2Drcxc3todWoBHTXN1vG299OaWwXxldAmDwQCcjs5ckcbo5I12nUPOOtGqLorHtrSsyX/JN/hYcWnCIYNR1DKyWKBwYPQWhlSZecKGyWhBJVJX6tbv989Ld6DuqUTTS27jP3K7JDKKfpXRVTK5JEe0iBQbJZN/Jzb2xPY+s4RnkP6TApsOKOheoTq4nMyPtaM0U1vzpX5uz5LG7HppZ7JNrJe0abaN6KDVNeahBNfFtweEthMfC1y52783zb8TmtDvV0fGY7ae/qOXUzhXnPgBQVdyJQs0zSxZkABfUJBf0w2XwzntukkTCD9nHp2p1cYjFHJ5NNaV6+ZS0CciNzSFxsNgf+bnW+2q2v2QVi4LP5cIBvcLen8CkW8rEQ==).

I have no insights in the underlying architecture, but I already have
**some comments on my PR** (and I am willing to work on these, if I get
some feedback/hints):

* the **decompression happens synchronously** (`zlib.inflateSync`). This
can be changed easily, once I understand how async operations are done
for the compiler explorer software.
* the link is `/zclientstate/` (in addition to `/clientstate/`) - I am
open for alternatives...
* I have **not added automated tests** (could be done, give me a
hint/example)

The modified SW worked for me! I hope to get some feedback... **This
would enable larger code-examples to be sent via clientstate as before
with smaller URLs**.

CONTRIBUTORS.md

@@ -150,3 +150,4 @@ From oldest to newest contributor, we would like to thank:
 - [Nazım Can Altınova](https://github.com/canova)
 - [Nicholas Hubbard](https://github.com/nhubbard)
 - [Detjon Mataj](https://github.com/detjonmataj)
+- [Pierre Bayerl](https://github.com/goto40)

docs/API.md

@@ -353,7 +353,9 @@ etcetera, otherwise <sourceid> can be set to 1.
 
 This call is to open the website with a given state (without having to store the state first with /api/shortener)
 Instead of sending the ClientState JSON in the post body, it will have to be encoded with base64 and attached directly
-onto the URL.
+onto the URL. It is possible to compress the JSON string with the zlib deflate method (compression used by gzip;
+available for many programming languages like [javascript](https://nodejs.org/api/zlib.html)). It is automatically
+detected.
 
 To avoid problems in reading base64 by the API, some characters must be kept in unicode. Therefore, before calling the
 API, it is necessary to replace these characters with their respective unicodes. A suggestion is to use the Regex

lib/handlers/route-api.ts

@@ -22,6 +22,8 @@
 // ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 // POSSIBILITY OF SUCH DAMAGE.
 
+import zlib from 'zlib';
+
 import express from 'express';
 
 import {AppDefaultArguments, CompilerExplorerOptions} from '../../app.js';
@@ -172,7 +174,7 @@ export class RouteAPI {
     }
 
     unstoredStateHandler(req: express.Request, res: express.Response) {
-        const state = JSON.parse(Buffer.from(req.params.clientstatebase64, 'base64').toString());
+        const state = extractJsonFromBufferAndInflateIfRequired(Buffer.from(req.params.clientstatebase64, 'base64'));
         const config = this.getGoldenLayoutFromClientState(new ClientState(state));
         const metadata = this.getMetaDataFromLink(req, null, config);
 
@@ -318,3 +320,17 @@ export class RouteAPI {
         return metadata;
     }
 }
+
+export function extractJsonFromBufferAndInflateIfRequired(buffer: Buffer): any {
+    const firstByte = buffer.at(0); // for uncompressed data this is probably '{'
+    const isGzipUsed = firstByte !== undefined && (firstByte & 0x0f) === 0x8; // https://datatracker.ietf.org/doc/html/rfc1950, https://datatracker.ietf.org/doc/html/rfc1950, for '{' this yields 11
+    if (isGzipUsed) {
+        try {
+            return JSON.parse(zlib.inflateSync(buffer).toString());
+        } catch (_) {
+            return JSON.parse(buffer.toString());
+        }
+    } else {
+        return JSON.parse(buffer.toString());
+    }
+}

test/handlers/route-api-test.ts

@@ -0,0 +1,65 @@
+// Copyright (c) 2024, Compiler Explorer Authors
+// All rights reserved.
+//
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions are met:
+//
+//     * Redistributions of source code must retain the above copyright notice,
+//       this list of conditions and the following disclaimer.
+//     * Redistributions in binary form must reproduce the above copyright
+//       notice, this list of conditions and the following disclaimer in the
+//       documentation and/or other materials provided with the distribution.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+// AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+// ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+// LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+// CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+// SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+// INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+// CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+// ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+// POSSIBILITY OF SUCH DAMAGE.
+
+import zlib from 'zlib';
+
+import {describe, expect, it} from 'vitest';
+
+import {extractJsonFromBufferAndInflateIfRequired} from '../../lib/handlers/route-api.js';
+
+function possibleCompression(buffer: Buffer): boolean {
+    // code used in extractJsonFromBufferAndInflateIfRequired
+    // required here to check criticality of test cases
+    const firstByte = buffer.at(0); // for uncompressed data this is probably '{'
+    return firstByte !== undefined && (firstByte & 0x0f) === 0x8; // https://datatracker.ietf.org/doc/html/rfc1950, https://datatracker.ietf.org/doc/html/rfc1950, for '{' this yields 11
+}
+
+describe('extractJsonFromBufferAndInflateIfRequired test cases', () => {
+    it('check that data extraction works (good case, no compression)', () => {
+        const buffer = Buffer.from('{"a":"test","b":1}');
+        expect(possibleCompression(buffer)).toBeFalsy();
+        const data = extractJsonFromBufferAndInflateIfRequired(buffer);
+        expect(data.a).toBe('test');
+        expect(data.b).toBe(1);
+    });
+    it('check that data extraction works (crirical case - first char indicates possible compression, no compression)', () => {
+        const buffer = Buffer.from('810');
+        expect(possibleCompression(buffer)).toBeTruthy();
+        const data = extractJsonFromBufferAndInflateIfRequired(buffer);
+        expect(data).toBe(810);
+    });
+    it('check that data extraction works (good case, with compression)', () => {
+        const text = '{"a":"test test test test test test test test test test test test test","b":1}';
+        const buffer = zlib.deflateSync(Buffer.from(text), {level: 9});
+        expect(buffer.length).lessThan(text.length);
+        expect(possibleCompression(buffer)).toBeTruthy();
+        const data = extractJsonFromBufferAndInflateIfRequired(buffer);
+        expect(data.a).toBe('test test test test test test test test test test test test test');
+        expect(data.b).toBe(1);
+    });
+    it('check that data extraction fails (bad case)', () => {
+        const buffer = Buffer.from('no json');
+        expect(() => extractJsonFromBufferAndInflateIfRequired(buffer)).toThrow();
+    });
+});