When a json string object is updated with a bigger string, a new
malloc'ed buffer is used to store the new string and it's size is made
negative to indicate that an external buffer is in use.
When that same json string object get's updated again with an empty
stirng (size = 0), the new external malloc'ed buffer is still used.
But the fact that the new size value is not negative removes the
indicator that the externally malloc'ed buffer is used.
This becomes a problem when the object get's updated again with any
other string, because a new buffer will be malloced and linked to the
object while to old one won't be free'd.
This causes a memory leak when updating a json string with
json_object_set_stirng() which has previously been updated
with an empty string.
Example:
--
obj = json_object_new_string("data");
json_object_set_string(obj, "more data");
json_object_set_string(obj, "");
json_object_set_string(obj, "other data"); /* leaks */
--
This commit fixes the issue by free'ing the external buffer when an
empty string is set and use the internal one again.
Signed-off-by: Daniel Danzberger <daniel@dd-wrt.com>
Use target_link_libraries, plus fill in Libs.private in json-c.pc so pkg-config --static --libs works appropriately.
Also, only link against libbsd when arc4random is actually found there.
If memory allocation fails in json_c_set_serialization_double_format or
json_object_copy_serializer_data then return with an error value and
preserve previous values without overriding them with NULL.
Do not silently truncate values or skip entries if out of memory errors
occur.
Proof of Concept:
- Create poc.c, a program which creates an eight megabyte large json
object with key "A" and a lot of "B"s as value, one of them is
UTF-formatted:
```c
#include <err.h>
#include <stdio.h>
#include <string.h>
#include "json.h"
#define STR_LEN (8 * 1024 * 1024)
#define STR_PREFIX "{ \"A\": \""
#define STR_SUFFIX "\\u0042\" }"
int main(void) {
char *str;
struct json_tokener *tok;
struct json_object *obj;
if ((tok = json_tokener_new()) == NULL)
errx(1, "json_tokener_new");
if ((str = malloc(STR_LEN)) == NULL)
err(1, "malloc");
memset(str, 'B', STR_LEN);
memcpy(str, STR_PREFIX, sizeof(STR_PREFIX) - 1);
memcpy(str + STR_LEN - sizeof(STR_SUFFIX), STR_SUFFIX, sizeof(STR_SUFFIX));
obj = json_tokener_parse(str);
free(str);
printf("%p\n", obj);
if (obj != NULL) {
printf("%.*s\n", 50, json_object_to_json_string(obj));
json_object_put(obj);
}
json_tokener_free(tok);
return 0;
}
```
- Compile and run poc, assuming you have enough free heap space:
```
gcc $(pkg-config --cflags --libs) -o poc poc.c
./poc
0x559421e15de0
{ "A": "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB
```
- Reduce available heap and run again, which leads to truncation:
```
ulimit -d 10000
./poc
0x555a5b453de0
{ "A": "B" }
```
- Compile json-c with this change and run with reduced heap again:
```
ulimit -d 10000
./poc
(nil)
```
The output is limited to 70 characters, i.e. json-c parses the 8 MB
string correctly but the poc does not print all of them to the screen.
The truncation occurs because the parser tries to add all chars up
to the UTF-8 formatted 'B' at once. Since memory is limited to 10 MB
there is not enough for this operation. The parser does not fail but
continues normally.
Another possibility is to create a json file close to 2 GB and run a
program on a system with limited amount of RAM, i.e. around 3 GB. But
ulimit restrictions are much easier for proof of concepts.
Treat memory errors correctly and abort operations.
If the input file is too large to fit into a printbuf then return an
error value instead of silently truncating the parsed content.
This introduces errno handling into printbuf to distinguish between an
input file being too large and running out of memory.
Most of these sites support HTTPS (some forward to HTTPS when accessing
the HTTP versions). Use HTTPS directly if supported.
Some URLs led to 404 error pages. Adjusted the links to point to
new locations.
I did not adjust the Microsoft HTML Help Workshop link because it seems
that this software is not available anymore. Instead of removing the
link entirely I kept it there in case it helps someone to find the
software on archived websites.
If errors occur in printbuf_memappend, then these errors should be
propagated through sprintbuf to indicate the error to the user.
Proof of Concept:
```
#include <err.h>
#include <limits.h>
#include <stdio.h>
#include "json.h"
int
main(void) {
struct printbuf *pb;
if ((pb = printbuf_new()) == NULL)
err(1, "printbuf_new");
if (printbuf_memset(pb, INT_MAX - 9, 'a', 1) < 0)
errx(1, "printbuf_memset");
printf("length: %d\n", printbuf_length(pb));
printf("sprintbuf: %d\n", sprintbuf(pb, "string too long"));
printf("length: %d\n", printbuf_length(pb));
printbuf_free(pb);
return 0;
}
```
You can see that sprintbuf does not return an error but length is still
the same, i.e. the string "string too long" has not been appended.
I would like to add this as a unit test but it really depends on the
operating system if printbuf_memset() would fail if not enough memory is
available or not.
It is possible to have a printbuf with "gaps", i.e. areas within the
print buffer which have not been initialized by using printbuf_memset.
Always clear memory in such cases.
Example:
```
struct printbuf *pb = printbuf_new();
printbuf_memset(pb, 10, 'a', 2);
```
In this case pb->buf[0] is '\0' but pb->buf[1] up to pb->buf[9] are
not set. The length would be 12 due to successful printbuf_memset.
Eric Haszlakiewicz
Daniel Danzberger
Tobias Stoeckmann
An7ar35
Julien Voisin
BonsaY