This lets us track things correctly when buffers are realloc()d. Rewrote fill() and append() to be more readable.