If memory allocation fails in json_c_set_serialization_double_format or
json_object_copy_serializer_data then return with an error value and
preserve previous values without overriding them with NULL.
Do not silently truncate values or skip entries if out of memory errors
occur.
Proof of Concept:
- Create poc.c, a program which creates an eight megabyte large json
object with key "A" and a lot of "B"s as value, one of them is
UTF-formatted:
```c
#include <err.h>
#include <stdio.h>
#include <string.h>
#include "json.h"
#define STR_LEN (8 * 1024 * 1024)
#define STR_PREFIX "{ \"A\": \""
#define STR_SUFFIX "\\u0042\" }"
int main(void) {
char *str;
struct json_tokener *tok;
struct json_object *obj;
if ((tok = json_tokener_new()) == NULL)
errx(1, "json_tokener_new");
if ((str = malloc(STR_LEN)) == NULL)
err(1, "malloc");
memset(str, 'B', STR_LEN);
memcpy(str, STR_PREFIX, sizeof(STR_PREFIX) - 1);
memcpy(str + STR_LEN - sizeof(STR_SUFFIX), STR_SUFFIX, sizeof(STR_SUFFIX));
obj = json_tokener_parse(str);
free(str);
printf("%p\n", obj);
if (obj != NULL) {
printf("%.*s\n", 50, json_object_to_json_string(obj));
json_object_put(obj);
}
json_tokener_free(tok);
return 0;
}
```
- Compile and run poc, assuming you have enough free heap space:
```
gcc $(pkg-config --cflags --libs) -o poc poc.c
./poc
0x559421e15de0
{ "A": "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
```
- Reduce available heap and run again, which leads to truncation:
```
ulimit -d 10000
./poc
0x555a5b453de0
{ "A": "B" }
```
- Compile json-c with this change and run with reduced heap again:
```
ulimit -d 10000
./poc
(nil)
```
The output is limited to 70 characters, i.e. json-c parses the 8 MB
string correctly but the poc does not print all of them to the screen.
The truncation occurs because the parser tries to add all chars up
to the UTF-8 formatted 'B' at once. Since memory is limited to 10 MB
there is not enough for this operation. The parser does not fail but
continues normally.
Another possibility is to create a json file close to 2 GB and run a
program on a system with limited amount of RAM, i.e. around 3 GB. But
ulimit restrictions are much easier for proof of concepts.
Treat memory errors correctly and abort operations.
If the input file is too large to fit into a printbuf then return an
error value instead of silently truncating the parsed content.
This introduces errno handling into printbuf to distinguish between an
input file being too large and running out of memory.
Most of these sites support HTTPS (some forward to HTTPS when accessing
the HTTP versions). Use HTTPS directly if supported.
Some URLs led to 404 error pages. Adjusted the links to point to
new locations.
I did not adjust the Microsoft HTML Help Workshop link because it seems
that this software is not available anymore. Instead of removing the
link entirely I kept it there in case it helps someone to find the
software on archived websites.
If errors occur in printbuf_memappend, then these errors should be
propagated through sprintbuf to indicate the error to the user.
Proof of Concept:
```
#include <err.h>
#include <limits.h>
#include <stdio.h>
#include "json.h"
int
main(void) {
struct printbuf *pb;
if ((pb = printbuf_new()) == NULL)
err(1, "printbuf_new");
if (printbuf_memset(pb, INT_MAX - 9, 'a', 1) < 0)
errx(1, "printbuf_memset");
printf("length: %d\n", printbuf_length(pb));
printf("sprintbuf: %d\n", sprintbuf(pb, "string too long"));
printf("length: %d\n", printbuf_length(pb));
printbuf_free(pb);
return 0;
}
```
You can see that sprintbuf does not return an error but length is still
the same, i.e. the string "string too long" has not been appended.
I would like to add this as a unit test but it really depends on the
operating system if printbuf_memset() would fail if not enough memory is
available or not.
It is possible to have a printbuf with "gaps", i.e. areas within the
print buffer which have not been initialized by using printbuf_memset.
Always clear memory in such cases.
Example:
```
struct printbuf *pb = printbuf_new();
printbuf_memset(pb, 10, 'a', 2);
```
In this case pb->buf[0] is '\0' but pb->buf[1] up to pb->buf[9] are
not set. The length would be 12 due to successful printbuf_memset.
Systems without vasprintf fall back to implementation in header file
vasprintf_compat.h. This version could run into heap overflow issues
with very long arguments or formats provoking a lot of output.
The vsnprintf function returns a negative value if more than INT_MAX
characters would be written since its int return type could not
handle this (and %n couldn't handle it either).
Before testing for a possible error value the additional char for
\0 is already added. A -1 error code would not be detected.
Increment only after implicitly casting to an unsigned value to avoid
signed integer overflow if INT_MAX has been returned.
Use va_copy to duplicate the original ap argument for multiple uses
on non-WIN32 systems. At least with glibc the test suite would fail
because the arguments are not reset after leaving the vsnprintf call.
Removed support for apparently very old glibc versions which do not
comply with vsnprintf standard descriptions. It breaks support for
modern ones which are not forced to return -1 in case of error. The
standard specifies merely "a negative value".
How to reproduce:
- Use a system without vasprintf
- Alternatively remove -D_GNU_SOURCE from CMakeLists.txt
- Compile and run:
#include "json.h"
int main(void) {
struct printbuf *pb = printbuf_new();
sprintbuf(pb, "prefix %2147483647s", "*");
printbuf_free(pb);
return 0;
}
The file was only be closed when there was no error and
was being left open when there was an error. By moving
the close(fd) statement out of the if-clause, the file
can be close regardless if there is an error or not.
After the file is closed, it can be checked for errors.
This makes Coverity Scan happier since it believes that the initial
check ``if (!src->_userdata && !src->_user_delete)`` could mean that
src->_user_data may be nullptr.
Current behaviour is perfectly valid, since wrap-over upon overflow is
well defined behaviour for unsigned types, but it is nevertheless nice to be
able to build with -fsanitize=undefined,unsigned-integer-overflow
There is no significant effect on the generated assembly as can be seen
on the diff of objdump -d output on a optimized build (the compiler
just decided to switch the order of a comparison):
@@ -135,8 +135,8 @@
1d0: 0f 84 70 ff ff ff je 146 <json_escape_str+0x146>
1d6: 4c 3b 24 24 cmp (%rsp),%r12
1da: 0f 85 2d ff ff ff jne 10d <json_escape_str+0x10d>
- 1e0: 49 39 f4 cmp %rsi,%r12
- 1e3: 0f 87 b7 00 00 00 ja 2a0 <json_escape_str+0x2a0>
+ 1e0: 4c 39 e6 cmp %r12,%rsi
+ 1e3: 0f 82 b7 00 00 00 jb 2a0 <json_escape_str+0x2a0>
1e9: 48 8b 44 24 18 mov 0x18(%rsp),%rax
1ee: 64 48 33 04 25 28 00 xor %fs:0x28,%rax
1f5: 00 00