json-c/bench/README.bench.md

Benchmark tests for json-c

General strategy:
-------------------

* Identify "after" commit hash
    * Use provided directory
    * Use provided commit hash
    * Local changes in current working directory
    * ${cur_branch}
* Identify "before" commit hash, in order of preference
    * Use provided directory
    * Use provided commit hash
    * Use origin/${cur_branch}, if different from ${after_commit}
    * Use previous release

* If not using existing dir, clone to src-${after_commit}
    * or, checkout appropriate commit in existing src-${after_commit}
* Create build & install dirs for ${after_commit}
* Build & install ${after_commit}
* Compile benchmark programs against install-${after_commit}

* If not using existing dir, clone to src-${before_commit}
    * or, checkout appropriate commit in existing src-${before_commit}
* Create build & install dirs for ${before_commit}
* Build & install ${before_commit}
* Compile benchmark programs against install-${before_commit}

* Run benchmark in each location
* Compare results

heaptrack memory profiler
---------------------------

https://milianw.de/blog/heaptrack-a-heap-memory-profiler-for-linux.html


```
yum install libdwarf-devel elfutils boost-devel libunwind-devel

git clone git://anongit.kde.org/heaptrack
cd heaptrack
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \
	-DCMAKE_INSTALL_PREFIX=$HOME/heaptrack-install ..
make install
```


Issues
--------

* jc-bench.sh is incomplete.

* valgrind massif misreports "extra-heap" bytes?

    "json_parse -n canada.json" shows 38640 KB maxrss.

    Using valgrind --tool=massif, a large amount of memory is listed as
     wasted "extra-heap" bytes.  (~5.6MB)

    ```
    valgrind --tool=massif --massif-out-file=massif.out ./json_parse -n ~/canada.json
    ms_print massif.out
    ```


    Using heaptrack, and analyzing the histogram, only shows ~2.6MB
    ```
    heaptrack ./json_parse -n canada.json
    heaptrack --analyze heaptrack*gz -H histogram.out
    awk ' { s=$1; count=$2; ru=(int((s+ 15) / 16)) * 16; wasted = ((ru-s)*count); print s, count, ru-s, wasted; total=total+wasted} END { print "Total: ", total }' histogram.out
    ```

 With the (unreleased) arraylist trimming changes, maxrss reported by
  getrusage() goes down, but massif claims *more* total usage, and a HUGE 
  extra-heap amount (50% of total).