Fix #2817: Add precision check to JsonTreeReader.nextInt() and nextLong()#3029
Fix #2817: Add precision check to JsonTreeReader.nextInt() and nextLong()#3029ErnestHysa wants to merge 2 commits into
Conversation
Both Gson.fromJson and JsonParser.parseReader accepted trailing data after a top-level null because the trailing-data check was skipped when the parsed element was null. Changed assertFullConsumption() in Gson.java to always check for END_DOCUMENT (removing the 'obj != null' guard), and changed the check in JsonParser.parseReader() to unconditionally verify END_DOCUMENT (removing the '!element.isJsonNull()' guard).
…nextLong() When a double value is stored in a JsonPrimitive, calling nextInt() or nextLong() should throw NumberFormatException to be consistent with JsonReader behavior, instead of silently truncating the value. This fix adds the same precision validation that JsonReader has: - nextInt() now checks if the double value can be exactly represented as an int, and throws if precision would be lost - nextLong() similarly checks for precision loss Fixes inconsistency where JsonReader fails but JsonTreeReader silently casts the value.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
|
CLA check looks like it didn't pick up my signature — I'm re-checking the linked Google account. The fix itself is ready for review on the code side; will update the CLA status as soon as it's cleared on the Google side. |
| private static void assertFullConsumption(Object obj, JsonReader reader) { | ||
| try { | ||
| if (obj != null && reader.peek() != JsonToken.END_DOCUMENT) { | ||
| if (reader.peek() != JsonToken.END_DOCUMENT) { |
There was a problem hiding this comment.
This accidentally includes changes from your other PR?
| JsonReader jsonReader = new JsonReader(reader); | ||
| JsonElement element = parseReader(jsonReader); | ||
| if (!element.isJsonNull() && jsonReader.peek() != JsonToken.END_DOCUMENT) { | ||
| if (jsonReader.peek() != JsonToken.END_DOCUMENT) { |
There was a problem hiding this comment.
This accidentally includes changes from your other PR?
|
Thanks for this! It's a behaviour change, but I think Something seems to have happened to the PR description. There's a lot of missing text, "calling or should throw to be consistent with behavior" and the correct and incorrect examples. Can you fix that? Along with the UnusedVariable finding. |
Marcono1234
left a comment
There was a problem hiding this comment.
Feel free to consider my comments only as suggestions, though I hope they are helpful.
@eamonnmcmanus, in #2031 (comment) you had concerns regarding related changes. But that PR changed JsonPrimitive behavior while this here only changes JsonTreeReader, so maybe this PR here is not as problematic?
| } | ||
| } | ||
| } else { | ||
| result = Long.parseLong(primitive.getAsString()); |
There was a problem hiding this comment.
Maybe this could instead still call getAsLong()?
| result = Long.parseLong(primitive.getAsString()); | |
| result = primitive.getAsLong(); |
| } | ||
| } | ||
| } else { | ||
| result = Integer.parseInt(primitive.getAsString()); |
There was a problem hiding this comment.
Maybe this could instead still call getAsInt()?
| result = Integer.parseInt(primitive.getAsString()); | |
| result = primitive.getAsInt(); |
| result = Long.parseLong(primitive.getAsString()); | ||
| } | ||
| } else { | ||
| throw new IllegalStateException("Unexpected token: " + o); |
There was a problem hiding this comment.
Might be better to use AssertionError to make it clear that this should be unreachable.
| throw new IllegalStateException("Unexpected token: " + o); | |
| throw new AssertionError("Unexpected token: " + o); |
| result = Integer.parseInt(primitive.getAsString()); | ||
| } | ||
| } else { | ||
| throw new IllegalStateException("Unexpected token: " + o); |
There was a problem hiding this comment.
Might be better to use AssertionError to make it clear that this should be unreachable.
| throw new IllegalStateException("Unexpected token: " + o); | |
| throw new AssertionError("Unexpected token: " + o); |
| if (number instanceof Long || number instanceof Integer) { | ||
| result = number.longValue(); | ||
| } else { |
There was a problem hiding this comment.
This will probably need explicit handling for BigInteger and BigDecimal (for nextLong() only) because values near Long.MAX_VALUE cannot be represented exactly by double.
So with your changes this would break behavior which was working previously for such values.
Maybe what you could do is:
- For
BigDecimal- call
BigDecimal#longValueExact() - catch the
ArithmeticExceptionand wrap it in a descriptiveNumberFormatException
- call
- For
BigInteger- check
bitLength() >= 64and throwNumberFormatExceptionin that case
Cannot callBigInteger#longValueExact()because it seems that method was only added recently to Android, for an API Level higher than Gson's minimum one. But maybe add a TODO comment to use that method in the future?
See also Issue 1994 #2031 (comment). - call
BigInteger#longValue()
- check
This is just a suggestion though. Maybe there are other ways to solve this as well.
| Number number = primitive.getAsNumber(); | ||
| if (number instanceof Long || number instanceof Integer) { | ||
| result = number.longValue(); | ||
| } else { |
There was a problem hiding this comment.
Similarly, might also need special handling for LazilyParsedNumber since that might also be common inside JsonPrimitive.
Maybe LazilyParsedNumber would need a new longValueExact() for that, maybe something like this?
public long longValueExact() {
try {
return Long.parseLong(value);
} catch (NumberFormatException ignored) {
BigDecimal bigDecimal = asBigDecimal();
try {
return bigDecimal.longValueExact();
} catch (ArithmeticException arithmeticException) {
NumberFormatException e = new NumberFormatException(arithmeticException.getMessage());
e.initCause(arithmeticException);
throw e;
}
}
}(but might have to catch the NumberFormatException here then, wrap it and include locationString()?)
Description
When a double value is stored in a JsonPrimitive, calling or should throw to be consistent with behavior, instead of silently truncating the value.
Issue
Issue #2817 reports that correctly throws when trying to read a double as an int/long, but silently casts the value.
JsonReader behavior (correct):
JsonTreeReader behavior (buggy):
Fix
This fix adds the same precision validation that has:
Changes
Testing
Added unit tests to verify that and throw when the stored value is a double that would lose precision when cast.
Fixes #2817