Help Center

Menu

Using attachments

Sometimes a binary attachment may need to be sent as part of an integration flow. Attachments may be used to:

  • Transfer product images together with the product description.

  • Transfer custom-encoded files before they can be parsed and transformed to JSON messages. For example the SFTP Component may read a CSV file without parsing it. The CSV component may parse it later and transform to JSON messages.

  • Preserve the native data format while working with it in components. For example, sometimes it’s required to work natively on XML without XML-JSON-XML transformation.

  • Handle large amounts of data as a single batch/bulk. Sometimes it’s required to batch a large number of messages and attachment storage may be a good accumulator to safely and efficiently stream to and stream from.

The attachment limit is 100MB. If you send bigger attachments or your flow generates bigger attachments you will get an error.

How attachments work

The Integrations uses an asynchronous message-oriented middleware (MOM) between the integration flow steps to ensure reliability and scalability. MOM is multi-tenant (in the multi-tenant plans) so quality of service and internal load balancing have to be ensured. It is much simpler to implement a flow if messages in the queues are roughly the same size so that broker don’t spend too much time on transferring a single multi-gigabyte message while thousands of smaller messages are waiting for their turn.

It is impractical to stream large payloads through MOM broker, therefore, binary attachments have to be offloaded to the external storage and linked in the message structure using the reference attachments.

EXAMPLE:

{
   "body": {
      "sku": "1234",
      "name": "MePhone Y Pro",
      "vendor": "Pear"
   },
   "attachments" : {
      "frontview.jpeg": {
         "content-type": "image/jpeg",
         "size": "45889",
         "url": "http://steward_host/files/1cfc3a71-d7a7-44e6-a15e-ae18860d537c"
      }
   }
}

As you can see, attachments are stored as a map, where keys are attachment names and values are defined as:

Value Description

content-type

(Optional). MIME content type of the attachment, will help you to get an initial idea of what could be inside.

size

(Optional) Size of the attachment in bytes.

url

HTTP URL where your component may download the attachment, you can be sure that URL contains all information you need to download it, no additional authentication should be required.

NOTE 1: You shouldn’t assume or expect anything about the format or host/port part of that URL, as it could be changed without the prior notice.

NOTE 2: Attachment URLs are internal to the platform, and cluster-specific. They cannot be accessed from outside of the cluster.

Creating new attachments

We have established how a component can access attachments. You can also receive/pull binary data and make it available as an attachment to the next component using the Connect API.

To create a new attachment you would need to do a HTTP POST to the following URL using authentication credentials that can be found in your container’s environment variables:

$ curl https://api.connect.squiz.cloud/v2/resources/storage/signed-url \
   -X POST \
   -u {EMAIL}:{APIKEY}

It will return something like this:

{
  "get_url": "http://steward_host/files/ea941d07-3ff5-4df1-b812-1bae2f0b9c36",
  "put_url": "http://steward_host/files/ea941d07-3ff5-4df1-b812-1bae2f0b9c36",
  "expires": 18000
}

Now you can use put_url to store your binary data and get_url you should place into the attachment section (as shown above) so that the next Component can read your binary data.