Ignore BOM char

[paulovieira]

Support plan

• is this issue currently blocking your project? (yes/no): yes
• is this issue affecting a production system? (yes/no): yes

Context

• node version: v16.14.0
• module version: 3.0.0
• environment (e.g. node, browser, native): node
• used with (e.g. hapi application, another framework, standalone, ...): wreck (with the json option)

What problem are you trying to solve?

In some weird cases the json response will have a byte order mark (BOM) character at the beginning. I experienced this when using @hapi/wreck to fetch data from a wacky API of a portuguese public service (see the example below).

Since bourne is meant to be a secure replacement for JSON.parse, I think it would make sense to detect and correct this well-known faulty case. In fact the JSON RFC suggests precisely that: https://datatracker.ietf.org/doc/html/rfc8259#section-8.1

Implementations MUST NOT add a byte order mark (U+FEFF) to the
beginning of a networked-transmitted JSON text. In the interests of
interoperability, implementations that parse JSON texts MAY ignore
the presence of a byte order mark rather than treating it as an
error.

The secure-json-parse module (which is the equivalent module in the fastify ecosystem) also does that: fastify/secure-json-parse#5

Below is a simple example that reproduces the problem. The response has a BOM char.

I'm happy to submit a PR if this is a desired feature for bourne.

const Wreck = require('@hapi/wreck');
const Bourne = require('@hapi/bourne');

const exampleThatWillFail = async function () {

    let { payload } = await Wreck.get(getUrl(), { json: true });
    console.log(payload)
};

const exampleThatWillSucceed = async function () {

    let { payload } = await Wreck.get(getUrl(), { json: false });

    // manually call Bourne.parse

    let s = payload.toString();
    if (s.charCodeAt(0) === 0xFEFF) {
        s = s.slice(1)
    }

    payload = Bourne.parse(s);
    console.log(payload)
};

try {
    exampleThatWillFail();
    // exampleThatWillSucceed();
}
catch (ex) {
    console.error(ex);
}

function getUrl() {

    return 'https://observatorioindicadores.dgterritorio.gov.pt/websig/bi/ngGeoAPI/public/index.php/metrics/load?par=observatorio&mod=metrics&type=0&table=t_new_observatorio_dat&identifier=0&query=and%20category_geo%20=%205%20and%20category_time%20=%201&classification=0&lang=pt&numclasses=5&precision=2&columns=category_time&rows=category_geo&ruddmode=&ruddparent=&ruddmember=&colors=%23FFF7EC%2C%23FEE8C8%2C%23FDD49E%2C%23FDBB84%2C%23FC8D59%2C%23EF6548%2C%23D7301F%2C%23B30000%2C%237F0000&param=894';
}

[Marsup]

As discussed in #32, I thought it was OK to do it, but reading more about it, I don't think we should accomodate invalid JSON. Stripping BOM should be your responsibility if you have a bad actor.