Review of proposed API

[ashcrow]

@jasinner / @abn,

I'd appreciate feedback on the new proposed API before I go much further implementing them 😄.

[jasinner]

We'll need a way to submit new vulnerability reports. Ideally we'd be able to do it by submitting a CVE in combination with:

• HASH
• (Maven) coordinates
• A file archive (which gets computed to a hash server side)

[abn]

In terms of submission this should be enabled today via https://github.com/victims/victims-web#secured-api-access - something on par should be considered.

HASH submissions should be treated with care as the fingerprint algorithm could have changed. That said, I am not sure if we should accept artifacts without coordinates? Note that regeneration becomes hard, and complexities are greater if we are dealing with hash submissions.

@ashcrow added some extra comments on the README. Can we put the content on some API doc service? Swagger/Apiary/readme.io etc? Might be a better way to hash out the APIs we need.

[ashcrow]

@abn yeah I'll have swagger enabled in the near future 😄

In terms of the secured-api-access I think we will drop the apikey/sig and just use basic auth due to feedback I've gotten from some users.

[ashcrow]

@jasinner ah, of course! Do you have an idea of how you'd like it to look or is cloning v2's upload process fine?

[abn]

@ashcrow I would say API key is way to go especially as submissions should really happen via clients. Was the feedback on the mailing list or in person? I'd prefer keys especially for automation scenarios too.

[jasinner]

I'd say the API key is OK, as long as provide examples of how to use it. I
found it challenging to get my head around it with the current API. I agree
that most submissions should come from automated clients.
I look forward to seeing the documentation generated by Swagger, as the
documentation for the old API was pretty slim. I'm still trying to figure
out how to use it.

On Thu, Sep 24, 2015 at 10:30 AM, Arun Babu Neelicattu <
notifications@github.com> wrote:

@ashcrow https://github.com/ashcrow I would say API key is way to go
especially as submissions should really happen via clients. Was the
feedback on the mailing list or in person? I'd prefer keys especially for
automation scenarios too.

—
Reply to this email directly or view it on GitHub
#1 (comment).

[ashcrow]

@abn they were in person similar to the complaints @jasinner noted above. Not well documented and/or more code than expected to make API calls.

Though I'm OK with keeping API as they way to access any non read-only endpoints.

[ashcrow]

@jasinner proposed upload endpoint added: https://github.com/ashcrow/victims-api/blob/master/ProposedAPI.md#submit. Let me know what you think.

[jasinner]

@ashcrow It seems it might not be compatible with Java submissions. If it's Java we need:

• groupId
• artifactId
• versionId
  This is handled by using http parameters per group in v2.
  The full list of requirements for submission groups can be found in the SUBMISSION_GROUPS value here:
  https://github.com/victims/victims-web/blob/master/src/victims_web/config.py

[ashcrow]

@jasinner how about the extra data that is different per group be part of the upload JSON? For a jar it could be something along the lines of:

{
    "groupId": "...",
    "artifactId": "..", 
    "versionId": "...",
}

while an egg would be:

{
    "package": "...",
}

Remind me, for a jar is versionId == version and artifactId == name? If so then it would just be groupId needed as extra data.

[jasinner]

@abn While we had this extra data in v2, I'm starting to wonder if it's really necessary. It only really matters that we have the file, and know which CVE it's vulnerable to. Why do we care about the library information? It could be inconsistent anyway, and sometimes, such as with many Red Hat libraries, they aren't found in Maven central.
Should we just leave this information out of a file submission, to keep it simple? I think it's better that we keep non-file vulnerability reports in https://github.com/victims/victims-cve-db, and use a trusted repo such as maven central as in index of library version to file data.
Perhaps we an offer another endpoint in v3 which only accepts library information, and looks up the file in Maven Central, or pep?

[ashcrow]

@jasinner as in query out to a maven repo or pypy for metadata?

[jasinner]

@ashcrow Not for metadata, but for the actually bits. We might be able to offer another endpoint which takes library metadata then looks those in up in Maven repo, or pypy for the corresponding file to hash.

[jasinner]

Actually, re-reading Arun's earlier comments regarding re-hashing we should be capturing the metadata so that we can rehash a submission if necessary. @ashcrow therefore we should capture information in JSON body, as you suggested. I think it's better to keep artifactId for Java. versionId, or version, either is fine.

That being said, it's going to be useless if metadata is not correct per the index (Maven central, or pypy). So we should aim to verify the metadata at submission time. Perhaps we can flag that the library was not found in the index to an admin before approving?