Stackhero S3 (Object Storage)

This add-on is operated by Stackhero

Object storage on dedicated instances with unlimited transfers

Stackhero S3 (Object Storage)

Last updated April 18, 2024

Stackhero Object Storage provides an object storage, based on MinIO, compatible with the S3 protocol and running on a fully dedicated instance.

It is easy to setup, fast and has a simple and predictive pricing. Especially, the network traffic is included and unlimited.

With your Stackhero Object Storage add-on you will get:

  • Unlimited transfers
  • A private instance (dedicated VM) for high performances and security
  • TLS encryption (aka SSL)
  • An automatic backup every 24 hours
  • A graphical web UI (MinIO console)
  • One click to update to new MinIO versions
  • Super fast servers with SSD disks

All S3 clients can connect to Stackhero Object Storage and there is a S3 client library for almost every language out there, including Ruby, Node.js, Java, Python, Clojure and Erlang.

You can also use the MinIO SDKs. This is the recommended way to connect to your object storage. You will find all the available SDKs on MinIO official documentation.

Another way to connect is to use the MinIO CLI interface. This is very helpful when you want to send big files directly from your terminal. You will find all the informations about it on the MinIO CLI official documentation.

Provisioning the add-on

Stackhero Object Storage can be attached to a Heroku application via the CLI:

A list of all plans available can be found here.

$ heroku addons:create ah-s3-object-storage-stackhero --app <your app name>
-----> Adding ah-s3-object-storage-stackhero to sharp-mountain-4005... done

After you provision Stackhero Object Storage, the STACKHERO_MINIO_ROOT_ACCESS_KEY and STACKHERO_MINIO_ROOT_SECRET_KEY config variables are available in your app’s configuration. They contain the “root” credentials.

You can see the content of those variables via the heroku config:get command:

$ heroku config:get STACKHERO_MINIO_ROOT_ACCESS_KEY
2765UKKnWOjjFtcXYSzW

We discourage users to directly use the “root” credentials in their applications. The good practice is to create a dedicated user and use it in your apps.

Create a user

A user is identified by an access key and a secret key that you will create.

There is two possibilities to create a user: use the MinIO web console or use the MinIO CLI client.

Create a user using the MinIO web console

To create a user on MinIO, you first have to connect to your Stackhero dashboard:

$ heroku addons:open ah-s3-object-storage-stackhero
Opening ah-s3-object-storage-stackhero for sharp-mountain-4005

Select your MinIO service and click on the Console link. Identify yourself with the STACKHERO_MINIO_ROOT_ACCESS_KEY and STACKHERO_MINIO_ROOT_SECRET_KEY credentials. You can retrieve them with the heroku config command or directly in Stackhero dashboard, by clicking on the Configure button.

Once connected to the MinIO console, click on “Users”, then “Create a user”. Define an access key, a secret key and select the policies you want (probably “readwrite”).

Creating a MinIO user

Once the user is created, you will be able to use these access key and secret key in your app.

Local setup

After you provision the add-on, it’s necessary to locally replicate its config variables so your development environment can operate against the service.

Use the Heroku Local command-line tool to configure, run and manage process types specified in your app’s Procfile. Heroku Local reads configuration variables from a .env file.

$ echo "STACKHERO_MINIO_ACCESS_KEY=XXXXXXX" >> .env
$ echo "STACKHERO_MINIO_SECRET_KEY=YYYYYYY" >> .env

Simply replace XXXXXXX per the access key value you have created before and YYYYYYY per the secret key.

Credentials and other sensitive configuration values should not be committed to source-control. In Git, exclude the .env file with: echo .env >> .gitignore.

For more information, see the Heroku Local article.

Using with Java

Here is an example of a connection using the MinIO SDK:

MinioClient minioClient =
  MinioClient.builder()
    .endpoint("https://" + System.getenv("STACKHERO_MINIO_HOST"))
    .credentials(System.getenv("STACKHERO_MINIO_ACCESS_KEY"), System.getenv("STACKHERO_MINIO_SECRET_KEY"))
    .build();

You will find more information on the official MinIO Java SDK documentation.

Using with Ruby

Install the AWS SDK gem:

$ gem 'aws-sdk'
$ bundle install

Here is a simple example of how to use MinIO via AWS SDK on Ruby:

require 'aws-sdk'

Aws.config.update(
  endpoint: 'https://' + ENV["STACKHERO_MINIO_HOST"],
  access_key_id: ENV["STACKHERO_MINIO_ACCESS_KEY"],
  secret_access_key: ENV["STACKHERO_MINIO_SECRET_KEY"],
  force_path_style: true,
  region: 'us-east-1'
)

rubys3_client = Aws::S3::Client.new

# put_object operation

rubys3_client.put_object(
  key: 'testobject',
  body: 'Hello from MinIO!!',
  bucket: 'testbucket',
  content_type: 'text/plain'
)

# get_object operation

rubys3_client.get_object(
  bucket: 'testbucket',
  key: 'testobject',
  response_target: 'download_testobject'
)

print "Downloaded 'testobject' as  'download_testobject'. "

You will find more information on the official MinIO Ruby documentation.

Using with Python

Using AWS SDK (boto)

Install the AWS SDK (boto) package:

$ pip install boto3
$ pip freeze > requirements.txt

Here is an example on how to connect to MinIO with Python and AWS SDK (boto) package:

#!/usr/bin/env/python
import os
import boto3
from botocore.client import Config

s3 = boto3.resource('s3',
                    endpoint_url='https://' + os.environ.get('STACKHERO_MINIO_HOST'),
                    aws_access_key_id=os.environ.get('STACKHERO_MINIO_ACCESS_KEY'),
                    aws_secret_access_key=os.environ.get('STACKHERO_MINIO_SECRET_KEY'),
                    config=Config(signature_version='s3v4'),
                    region_name='us-east-1')

# upload a file from local file system '/home/john/piano.mp3' to bucket 'songs' with 'piano.mp3' as the object name.
s3.Bucket('songs').upload_file('/home/john/piano.mp3','piano.mp3')

# download the object 'piano.mp3' from the bucket 'songs' and save it to local FS as /tmp/classical.mp3
s3.Bucket('songs').download_file('piano.mp3', '/tmp/classical.mp3')

print "Downloaded 'piano.mp3' as  'classical.mp3'. "

You will find more information on the official MinIO Python documentation.

Using MinIO SDK

Install the MinIO package:

$ pip install minio
$ pip freeze > requirements.txt

Here is an example on how to connect to MinIO with Python:

#!/usr/bin/env/python
from minio import Minio
from minio.error import S3Error

def main():
  # Create a client with the MinIO server playground, its access key
  # and secret key.
  client = Minio(
    endpoint=os.environ.get('STACKHERO_MINIO_HOST') + ':443,
    secure=True,
    access_key=os.environ.get('STACKHERO_MINIO_ACCESS_KEY'),
    secret_key=os.environ.get('STACKHERO_MINIO_SECRET_KEY'),
  )

  # Make 'asiatrip' bucket if not exist.
  found = client.bucket_exists("asiatrip")
  if not found:
    client.make_bucket("asiatrip")
  else:
    print("Bucket 'asiatrip' already exists")

  # Upload '/home/user/Photos/asiaphotos.zip' as object name
  # 'asiaphotos-2015.zip' to bucket 'asiatrip'.
  client.fput_object(
    "asiatrip", "asiaphotos-2015.zip", "/home/user/Photos/asiaphotos.zip",
  )
  print(
    "'/home/user/Photos/asiaphotos.zip' is successfully uploaded as "
    "object 'asiaphotos-2015.zip' to bucket 'asiatrip'."
  )


if __name__ == "__main__":
  try:
    main()
  except S3Error as exc:
    print("error occurred.", exc)

You will find more information on the official MinIO Python SDK documentation.

Using with PHP

Install the AWS SDK for PHP package:

$ composer require aws/aws-sdk-php

Here is an example on how to connect to MinIO with PHP and AWS SDK:

<?php

// Include the SDK using the Composer autoloader
date_default_timezone_set('America/Los_Angeles');
require 'vendor/autoload.php';

$s3 = new Aws\S3\S3Client([
  'version' => 'latest',
  'region'  => 'us-east-1',
  'endpoint' => 'https://' . getenv('STACKHERO_MINIO_HOST'),
  'use_path_style_endpoint' => true,
  'credentials' => [
    'key'    => getenv('STACKHERO_MINIO_ACCESS_KEY'),
    'secret' => getenv('STACKHERO_MINIO_SECRET_KEY')
  ],
]);


// Send a PutObject request and get the result object.
$insert = $s3->putObject([
  'Bucket' => 'testbucket',
  'Key'    => 'testkey',
  'Body'   => 'Hello from MinIO!!'
]);

// Download the contents of the object.
$retrive = $s3->getObject([
  'Bucket' => 'testbucket',
  'Key'    => 'testkey',
  'SaveAs' => 'testkey_local'
]);

// Print the body of the result by indexing into the result object.
echo $retrive['Body'];

You will find more information on the official MinIO PHP documentation.

Using with Go

Using AWS SDK

Install the AWS SDK package for GO:

$ go get github.com/aws/aws-sdk-go-v2
$ go get github.com/aws/aws-sdk-go-v2/config

Here is an example on how to connect to MinIO with GO and AWS SDK:

package main

import (
  "fmt"
  "os"
  "strings"

  "github.com/aws/aws-sdk-go/aws"
  "github.com/aws/aws-sdk-go/aws/credentials"
  "github.com/aws/aws-sdk-go/service/s3/s3manager"
  "github.com/aws/aws-sdk-go/aws/session"
  "github.com/aws/aws-sdk-go/service/s3"
)

func main() {
  bucket := aws.String("newbucket")
  key := aws.String("testobject")

  // Configure to use MinIO Server
  s3Config := &aws.Config{
    Credentials:      credentials.NewStaticCredentials(os.Getenv("STACKHERO_MINIO_ACCESS_KEY"), os.Getenv("STACKHERO_MINIO_SECRET_KEY"), ""),
    Endpoint:         aws.String("https://" + os.Getenv("STACKHERO_MINIO_HOST")),
    Region:           aws.String("us-east-1"),
    DisableSSL:       aws.Bool(false),
    S3ForcePathStyle: aws.Bool(true),
  }
  newSession := session.New(s3Config)

  s3Client := s3.New(newSession)

  cparams := &s3.CreateBucketInput{
    Bucket: bucket, // Required
  }

  // Create a new bucket using the CreateBucket call.
  _, err := s3Client.CreateBucket(cparams)
  if err != nil {
    // Message from an error.
    fmt.Println(err.Error())
    return
  }

  // Upload a new object "testobject" with the string "Hello World!" to our "newbucket".
  _, err = s3Client.PutObject(&s3.PutObjectInput{
    Body:   strings.NewReader("Hello from MinIO!!"),
    Bucket: bucket,
    Key:    key,
  })
  if err != nil {
    fmt.Printf("Failed to upload data to %s/%s, %s\n", *bucket, *key, err.Error())
    return
  }
  fmt.Printf("Successfully created bucket %s and uploaded data with key %s\n", *bucket, *key)

  // Retrieve our "testobject" from our "newbucket" and store it locally in "testobject_local".
  file, err := os.Create("testobject_local")
  if err != nil {
    fmt.Println("Failed to create file", err)
    return
  }
  defer file.Close()

  downloader := s3manager.NewDownloader(newSession)
  numBytes, err := downloader.Download(file,
  &s3.GetObjectInput{
    Bucket: bucket,
    Key:    key,
  })
  if err != nil {
    fmt.Println("Failed to download file", err)
    return
  }
  fmt.Println("Downloaded file", file.Name(), numBytes, "bytes")
}

You will find more information on the official MinIO GO documentation.

Using MinIO SDK

Install the MinIO SDK package for GO:

$ GO111MODULE=on go get github.com/minio/minio-go/v7

Here is an example on how to connect to MinIO with GO and MinIO SDK:

package main

import (
  "log"

  "github.com/minio/minio-go/v7"
  "github.com/minio/minio-go/v7/pkg/credentials"
)

func main() {
  endpoint := os.Getenv("STACKHERO_MINIO_HOST")
  accessKeyID := os.Getenv("STACKHERO_MINIO_ACCESS_KEY")
  secretAccessKey := os.Getenv("STACKHERO_MINIO_SECRET_KEY")
  useSSL := true


  // Initialize minio client object.
  minioClient, err := minio.New(endpoint, &minio.Options{
    Creds:  credentials.NewStaticV4(accessKeyID, secretAccessKey, ""),
    Secure: useSSL,
  })
  if err != nil {
    log.Fatalln(err)
  }

  log.Printf("%#v\n", minioClient) // minioClient is now set up
}

You will find more information on the official MinIO GO SDK documentation.

Using with Node.js

Using AWS SDK

Install the AWS SDK package for Node.js:

$ npm install --save aws-sdk

Here is an example on how to connect to MinIO with Node.js and MinIO SDK:

const process = require('process');
const AWS = require('aws-sdk');

const s3  = new AWS.S3({
  accessKeyId: process.env.STACKHERO_MINIO_ACCESS_KEY,
  secretAccessKey: process.env.STACKHERO_MINIO_SECRET_KEY,
  endpoint: 'https://' + process.env.STACKHERO_MINIO_HOST,
  s3ForcePathStyle: true, // needed with minio?
  signatureVersion: 'v4'
});

// putObject operation.
s3.putObject(
  { Bucket: 'testbucket', Key: 'testobject', Body: 'Hello from MinIO!!' },
  (err, data) => {
    if (err)
      console.log(err)
    else
      console.log('Successfully uploaded data to testbucket/testobject');
  }
);

// getObject operation.
const file = require('fs').createWriteStream('/tmp/mykey');
s3.getObject({ Bucket: 'testbucket', Key: 'testobject' })
  .on('httpData', chunk => file.write(chunk))
  .on('httpDone', () => file.end())
  .send();

You will find more information on the official MinIO Node.js documentation.

Using MinIO SDK

Install the MinIO SDK package for Node.js:

$ npm install --save minio

Here is an example on how to connect to MinIO with Node.js and MinIO SDK:

const process = require('process');
const Minio = require('minio');

// Instantiate the minio client with the endpoint
// and access keys as shown below.
const minioClient = new Minio.Client({
  endpoint: process.env.STACKHERO_MINIO_HOST,
  port: 443,
  useSSL: true,
  accessKeyId: process.env.STACKHERO_MINIO_ACCESS_KEY,
  secretAccessKey: process.env.STACKHERO_MINIO_SECRET_KEY,
});

// File that needs to be uploaded.
const file = '/tmp/photos-europe.tar'

// Make a bucket called europetrip.
minioClient.makeBucket(
  'europetrip',
  'us-east-1',
  err => {
    if (err) return console.log(err)

    console.log('Bucket created successfully in "us-east-1".')

    const metaData = {
      'Content-Type': 'application/octet-stream',
      'X-Amz-Meta-Testing': 1234,
      'example': 5678
    }
    // Using fPutObject API upload your file to the bucket europetrip.
    minioClient.fPutObject(
      'europetrip',
      'photos-europe.tar',
      file,
      metaData,
      (err, etag) => {
        if (err) return console.log(err)
        console.log('File uploaded successfully.')
      }
    );
  }
);

You will find more information on the official MinIO Node.js SDK documentation.

Dashboard and MinIO console web UI

Stackhero dashboard allows you to see your instance usage, restart it, and apply updates. It also gives you the ability to access the MinIO console web UI to consult your MinIO data directly in a graphical way.

You can access the dashboard via the CLI:

$ heroku addons:open ah-s3-object-storage-stackhero
Opening ah-s3-object-storage-stackhero for sharp-mountain-4005

or by visiting the Heroku Dashboard and selecting the application in question. Select “Stackhero Object Storage from the Add-ons menu.

Upgrading your plan

You cannot downgrade an existing add-on.

Application owners should carefully manage the migration timing to ensure proper application function during the migration process.

Use the heroku addons:upgrade command to migrate to a new plan.

$ heroku addons:upgrade ah-s3-object-storage-stackhero:newplan
-----> Upgrading ah-s3-object-storage-stackhero:newplan to sharp-mountain-4005... done
       Your plan has been updated to: ah-s3-object-storage-stackhero:newplan

Removing the add-on

You can remove Stackhero Object Storage via the CLI:

This will destroy all associated data and cannot be undone!

$ heroku addons:destroy ah-s3-object-storage-stackhero
-----> Removing ah-s3-object-storage-stackhero from sharp-mountain-4005... done

Support

Stackhero Object Storage support and runtime issues should be submitted via one of the Heroku Support channels. We recommend adding support@stackhero.io in copy for urgent issues.

Additional resources

Feedback

Log in to submit feedback.