将字节数组转换为base128有效的JSON字符串 [英] Convert array of bytes to base128 valid JSON string

问题描述

我想使用JSON发送大量字节（我的灵感来自于这个问题），有一个小开销，我想使用base128编码（实际上可以产生有效的json字符串）。但不幸的是，我无法找到一些在JS中进行转换的程序。我将发布我的程序作为这个问题的答案，但是可能有人有更短的程序或者可能更好的想法在JSON中有效地发送二进制数据。

I want to send big array of bytes using JSON (I was inspired by this question), to have small overhead I wana to use base128 encoding (which can in fact produce valid json string). But unfortunately I was unable to find some procedures which do that conversions in JS. I will publish my procedures as answer to this question, however may be someone has shorter procedures or may be better idea to effective sending binary data inside JSON.

推荐答案

ES6：

编码

ES6:

Encode

let bytesToBase128 = (bytesArr) => {
    // 128 characters to encode as json-string
    let c= "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz¼½ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ" 
    let fbits=[]; 
    let bits = (n,b=8) => [...Array(b)].map((x,i)=>n>>i&1); 
    bytesArr.map(x=> fbits.push(...bits(x)));

    let fout=[]; 
    for(let i =0; i<fbits.length/7; i++) { 
        fout.push(parseInt(fbits.slice(i*7, i*7+7).reverse().join(''),2))  
    }; 

    return (fout.map(x => c[x])).join('');
}

// Example
// bytesToBase128([23, 45, 65, 129, 254, 42, 1, 255]) => "NÚ4AèßÊ0ÿ1"

解码

let base128ToBytes = (base128str) => {
    // 128 characters to encode as json-string
    let c= "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz¼½ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ" 

    dfout = base128str.split('').map(x=>c.indexOf(x));
    let dfbits = [];
    let bits = (n,b=8) => [...Array(b)].map((x,i)=>n>>i&1);
    dfout.map(x=> dfbits.push(...bits(x,7) ));

    let dfbytes=[]; 
    let m1 = dfbits.length%8 ? 1 : 0;
    for(let i =0; i<dfbits.length/8-m1; i++) { 
        dfbytes.push(parseInt(dfbits.slice(i*8, i*8+8).reverse().join(''),2))  
    }; 

    return dfbytes;
}

// Example
// base128ToBytes("NÚ4AèßÊ0ÿ1") => [23, 45, 65, 129, 254, 42, 1, 255]

我在这里嵌入位功能 - 此处。这里的转换思想是将字节数组转换为位数组，然后将每个7位（值为0到127）作为字符编号i charakter list c 。在解码中，我们更改每个字符数7位数并创建数组，然后获取此数组的每个8位包并将它们解释为byte。

I embeded here bits function - here. The coversion idea here is to convert bytes array to bit array and then take each 7 bits (the value is from 0 to 127) as character number i charakter list c. In decoding we change each character number 7-bit number and create array, and then take each 8-bit packages of this array and interpret them as byte.

查看字符来自ASCI并从中选择128（这是任意的）我输入控制台

To view characters from ASCI and choose 128 from them (which is arbitrary) I type in console

[...Array(256)].map((x,i) => String.fromCharCode(i)).join('');

我尽量避免在等不同情境中具有特殊含义的字符！ @＃$％'& ...

这里是工作示例（将 Float32Array 转换为json）。

And here is working example (which convert Float32Array to json).

在Chrome，Firefox上测试和Safari

Tested on Chrome, Firefox and Safari

将字节数组转换为base128字符串（有效的json）后，输出字符串为小于 15％大于输入数组。

After conversion bytes array to base128 string (which is valid json) the output string is less than 15% bigger than input array.

再多挖一点，并且当我们发送代码大于128的字符（¼½ÀÁÂÃ...... ... ）时，实际上发送了两个字符（而不是一个:( - 我以这种方式进行测试：在URL栏中输入 chrome：// net-internals /＃events （并发送POST请求）和 URL_REQUEST> HTTP_STREAM_REQUEST> UPLOAD_DATA_STREAM_INIT> total_size 当身体包含大于128的字符代码时，我们看到该请求是两倍大。所以实际上我们没有profi发送此字符:(。对于base64字符串，我们没有观察到这样的负面行为 - 但是我离开了这个程序，因为它们可能被用于其他目的而不是发送（比如在localstorage中存储二进制数据比base64更好的替代 - 但是可能存在更好的方法......？） 。 更新2019年 此处。

A dig a little bit more, and expose that when we send characters which codes are bigger than 128 (¼½ÀÁÂÃÄ...) then chrome in fact send TWO characters (bytes) instead one :( - I made test in this way: type in url bar chrome://net-internals/#events (and send POST request) and in URL_REQUEST> HTTP_STREAM_REQUEST > UPLOAD_DATA_STREAM_INIT > total_size we see that request are two times bigger when body contains charaters witch codes bigger than 128. So in fact we don't have profit with sending this characters :( . For base64 strings we not observe such negative behaviour - However I left this procedures because they may bye used in other purposes than sending (like better alternative to storage binary data in localstorage than base64 - however probably there exists even better ways...?). UPDATE 2019 here.

这篇关于将字节数组转换为base128有效的JSON字符串的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！