Fog and S3 multipart uploads

Digging through issues and source code

I was hitting Excon::Errors::SocketError: Broken pipe (Errno::EPIPE) when trying to upload relatively small file (30MB) to S3. The most common suggestion is to set region when creating Fog’s storage object.
  provider:              'AWS',
  aws_access_key_id:     aws_access_key_id,
  aws_secret_access_key: aws_secret_access_key,
  region:                'us-east-1',

Unfortunately this didn’t help.

To allieviate that problem I decided to go route of S3 multipart uploads, which Fog supports. The problem it’s not exposed in any friendly way. To get multipart uploads you need to do two things: make sure body is a file like object (with at least #read method) and to set multipart_chunk_size on the file which you want to save.

file =
  key: filename,
file.multipart_chunk_size = 5242880  # 5MB

When those two conditions are met fog will happily upload file using S3 multipart upload, which in my case eliminated those broken pipe errors.