Streaming libvips

At the moment, all existing popular Go + libvips image applications (imgproxy, imaginary, bimg etc) bridge libvips through buffer. While these are all good with normal web images, the latency and memory overhead can be noticeable when working through large, raw images, as they are all loading the whole image buffer in memory, one step at a time.

Imagor v1 fully revamped the libvips binding, now implemented its streaming capability that enables parallel processing across Loader, Storage and Processor. This greatly increases network throughput especially with network sources like HTTPs, S3, GCS, by giving the ability to overlap processing pipelines.

Metadata and Exif

Imagor now provides metadata endpoint that extracts image info such as image format, dimensions and Exif metadata. With streaming in place, same goes for image Exif metadata. Imagor can try to retrieve data just enough to extract the header, without reading and processing the whole image in memory.

To use the metadata endpoint, add /meta right after the URL signature hash before the image operations. Example:

http://localhost:8000/unsafe/meta/fit-in/50x50/raw.githubusercontent.com/cshum/imagor/master/testdata/Canon_40D.jpg

{
  "format": "jpeg",
  "content_type": "image/jpeg",
  "width": 50,
  "height": 34,
  "orientation": 1,
  "pages": 1,
  "exif": {
    "ApertureValue": "368640/65536",
    "ColorSpace": 1,
    "ComponentsConfiguration": "Y Cb Cr -",
    "Compression": 6,
    "DateTime": "2008:07:31 10:38:11",
    // ...
    "WhiteBalance": 0,
    "XResolution": "72/1",
    "YCbCrPositioning": 2,
    "YResolution": "72/1"
  }
}

What's Changed

• feat(vips): revamp libvips binding by @cshum
• feat(vips): vips load image directly from file by @cshum
• build(docker): fix docker build steps by @cshum
• feat(vips): true streaming for libvips by @cshum
• refactor(vips): context gc tracking by @cshum
• build(deps): bump github.com/peterbourgon/ff/v3 from 3.2.0-rc.1 to 3.3.0 by @dependabot
• build(deps): bump github.com/aws/aws-sdk-go from 1.44.66 to 1.44.70 by @dependabot
• refactor(imagor): improve processor and storage concurrency by @cshum
• feat(vips): exif metadata support by @cshum
• refactor: simplify metadata architecture by @cshum

Full Changelog: v0.9.13...v1.0.0