Setting Up Kaltura S3 Cloudfront CDN – CE 9 External Storage With CDN Delivery

Hi,

Today I was asked by one of our clients to install a Kaltura cluster with Amazon S3 storage Cloudfront CDN. Kaltura S3 Cloudfront is a popular setup among our customers. For assistance, I used our old blogpost (posted about a year ago): SETTING UP KALTURA CE 5.0 AMAZON S3 STORAGE CLOUDFRONT CDN – EXTERNAL STORAGE WITH CDN DELIVERY which explains very well how to set up Kaltura CE Amazon S3 Storage and Cloudfront CDN. Although it was very helpful, I found a bit of a difference between Kaltura CE 5.0 and CE 9.X versions. I thought that an updated post regarding this issue would be a good idea.

Kaltura CE Amazon S3 Storage CloudFront CDN Integration

The purpose of this post is to help you set up the external storage in Kaltura CE 9.X, to work with Amazon S3 and Cloudfront CDN. Keyword: Kaltura CE Amazon S3 Cloudfront CDN

Kaltura CE 9.X and Amazon S3 external storage

The Kaltura Community Edition (CE) 9X comes with built-in support for external storage with Amazon Simple Storage Service (S3). It seemed at first that the integration would be simple and straightforward. Install Kaltura CE, set up the S3 bucket, create a remote distribution profile, and enable the batch job to run the sync. However, I ran into several problems in the Kaltura configuration. I wish to share my findings and my solution to a working installation of Kaltura CE 9.X, serving all content from an S3 bucket via the Amazon Cloudfront CDN.

Setting up Amazon S3 and getting security credentials

1. To get your Amazon security credentials (assuming you have an account with amazon AWS), go to this link https://portal.aws.amazon.com/gp/aws/securityCredentials

2. To set up your amazon S3 bucket, go to Amazon S3, create a new bucket, and name it.

3. Select your new bucket on the left side, click Actions and select “Properties”.

4. Add more permissions – Authenticated Users – check all boxes.

5. Inside this bucket, create a folder called “kaltura” (or any other name you wish).

6. Add more permissions – select the “kaltura” folder, right click the folder and select “Make Public”.

Notice: Making the folder public will only apply on the existing files in the folder and any new file added will not get the required permission, and must be configured manually.
In order to set permissions to newly uploaded files, you need to generate a permission policy for the entire bucket and set it for the bucket.
Information and examples of S3 bucket policies: http://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html

Setting up Amazon CloudFront CDN

1. Go to https://console.aws.amazon.com/cloudfront/home

2. Create a new “Distribution” of “Web” type delivery method and name it.

3. Select your bucket as the origin ID, and configure other properties.

4. Copy your CloudFront domain name (example: zdfx73d6dzff32z.cloudfront.net) for later use.

Setting up the Remote Storage Profile in the Admin Console

First, you must enable the necessary configuration options for your partner:

1. Find your partner in the list of partners, click on the right drop down box and select “Configure”.

2. Under “Remote Storage Policy”, set Delivery Policy to “Remote Storage Only” (or other option as you decide).

3. Check the “Delete exported storage” checkbox.

4. Under Enable/Disable Features, make sure that “Remote Storage” is checked.

5. Click “Save”.

Next we must configure the Remote Storage Profile. In order to do this, we must click on the partner’s left drop-down box (under “Profiles”) and select “Remote Storage”. You should see the “Remote Storage Profiles” page for your publisher (If you haven’t yet set up any remote storage profiles, the list should be empty).

(Assuming that you have already set up an S3 bucket, and that you have an Access Key ID and a Secret Access Key)

1. Create a new profile by writing your publisher id in the right “Publisher ID” input box and clicking “Create New”.

2. Give a name to your Remote Storage (for example “Amazon S3”)

3. For “Storage URL” type http://{yourbucketname}.s3.amazonaws.com (replace {yourbucketname} with your bucket name on S3)

4. In Storage Base Directory, write “/{yourbucketname}/kaltura” (keep in mind the leading slash, and change yourbucketname to your bucket name)

5. Storage Username – enter your amazon aws api Access Key ID

6. Storage Password – paste your amazon aws api Secret Access Key

7. Under HTTP Delivery Base URL, type “http://{your amazon cloudfront domain}/kaltura” – replace {your amazon cloudfront domain} with the cloudfront domain you created in the previous section).

8. Save the new Remote Storage Profile

Add a crossdomain.xml file
A cross domain file is an XML document that grants a web client—such as Adobe Flash Player, Adobe Reader and others, permission to handle data across multiple domains.
Create a crossdomain.xml file in the root of your S3 bucket:

<cross-domain-policy xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://www.adobe.com/xml/schemas/PolicyFile.xsd">
    <allow-access-from domain="*" to-ports="*" secure="false"/>
    <site-control permitted-cross-domain-policies="all"/>
    <allow-http-request-headers-from domain="*" headers="*"/>
</cross-domain-policy>

Final Step – Enable the remote storage profile
1. Click on the dropdown box next to your new storage profile in the Remote Storage Profiles page in Kaltura Admin Console

2. Select “Export Automatically” and then click “OK”

3. You will receive the confirmation that your storage was autoed 🙂

Test your new configuration

You can go ahead and test your new configuration. Upload a new video in the KMC, let it convert, and wait for it to get distributed. After that, try to play the entry and analyze it in your favorite sniffer. You should see that the movies are being downloaded from your cloudfront CDN, look for flv and mp4 files.

Good Luck

Vadim Tarasov

Software developer, PandaOS