Setting Up Kaltura S3 Cloudfront CDN – CE 9 External Storage With CDN Delivery

Hi,

Today I was asked by one of our clients to install a Kaltura cluster with Amazon S3 storage Cloudfront CDN. Kaltura S3 Cloudfront is a popular setup among our customers. For assistance, I used our old blogpost (posted about a year ago): SETTING UP KALTURA CE 5.0 AMAZON S3 STORAGE CLOUDFRONT CDN – EXTERNAL STORAGE WITH CDN DELIVERY which explains very well how to set up Kaltura CE Amazon S3 Storage and Cloudfront CDN. Although it was very helpful, I found a bit of a difference between Kaltura CE 5.0 and CE 9.X versions. I thought that an updated post regarding this issue would be a good idea.

Kaltura CE Amazon S3 Storage CloudFront CDN Integration

The purpose of this post is to help you set up the external storage in Kaltura CE 9.X, to work with Amazon S3 and Cloudfront CDN. Keyword: Kaltura CE Amazon S3 Cloudfront CDN

Kaltura CE 9.X and Amazon S3 external storage

The Kaltura Community Edition (CE) 9X comes with built-in support for external storage with Amazon Simple Storage Service (S3). It seemed at first that the integration would be simple and straightforward. Install Kaltura CE, set up the S3 bucket, create a remote distribution profile, and enable the batch job to run the sync. However, I ran into several problems in the Kaltura configuration. I wish to share my findings and my solution to a working installation of Kaltura CE 9.X, serving all content from an S3 bucket via the Amazon Cloudfront CDN.

Setting up Amazon S3 and getting security credentials

1. To get your Amazon security credentials (assuming you have an account with amazon AWS), go to this link https://portal.aws.amazon.com/gp/aws/securityCredentials

2. To set up your amazon S3 bucket, go to Amazon S3, create a new bucket, and name it.

3. Select your new bucket on the left side, click Actions and select “Properties”.

4. Add more permissions – Authenticated Users – check all boxes.

5. Inside this bucket, create a folder called “kaltura” (or any other name you wish).

6. Add more permissions – select the “kaltura” folder, right click the folder and select “Make Public”.

Notice: Making the folder public will only apply on the existing files in the folder and any new file added will not get the required permission, and must be configured manually.
In order to set permissions to newly uploaded files, you need to generate a permission policy for the entire bucket and set it for the bucket.
Information and examples of S3 bucket policies: http://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html

Setting up Amazon CloudFront CDN

1. Go to https://console.aws.amazon.com/cloudfront/home

2. Create a new “Distribution” of “Web” type delivery method and name it.

3. Select your bucket as the origin ID, and configure other properties.

4. Copy your CloudFront domain name (example: zdfx73d6dzff32z.cloudfront.net) for later use.

Setting up the Remote Storage Profile in the Admin Console

First, you must enable the necessary configuration options for your partner:

1. Find your partner in the list of partners, click on the right drop down box and select “Configure”.

2. Under “Remote Storage Policy”, set Delivery Policy to “Remote Storage Only” (or other option as you decide).

3. Check the “Delete exported storage” checkbox.

4. Under Enable/Disable Features, make sure that “Remote Storage” is checked.

5. Click “Save”.

Next we must configure the Remote Storage Profile. In order to do this, we must click on the partner’s left drop-down box (under “Profiles”) and select “Remote Storage”. You should see the “Remote Storage Profiles” page for your publisher (If you haven’t yet set up any remote storage profiles, the list should be empty).

(Assuming that you have already set up an S3 bucket, and that you have an Access Key ID and a Secret Access Key)

1. Create a new profile by writing your publisher id in the right “Publisher ID” input box and clicking “Create New”.

2. Give a name to your Remote Storage (for example “Amazon S3”)

3. For “Storage URL” type http://{yourbucketname}.s3.amazonaws.com (replace {yourbucketname} with your bucket name on S3)

4. In Storage Base Directory, write “/{yourbucketname}/kaltura” (keep in mind the leading slash, and change yourbucketname to your bucket name)

5. Storage Username – enter your amazon aws api Access Key ID

6. Storage Password – paste your amazon aws api Secret Access Key

7. Under HTTP Delivery Base URL, type “http://{your amazon cloudfront domain}/kaltura” – replace {your amazon cloudfront domain} with the cloudfront domain you created in the previous section).

8. Save the new Remote Storage Profile

Add a crossdomain.xml file
A cross domain file is an XML document that grants a web client—such as Adobe Flash Player, Adobe Reader and others, permission to handle data across multiple domains.
Create a crossdomain.xml file in the root of your S3 bucket:

<cross-domain-policy xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://www.adobe.com/xml/schemas/PolicyFile.xsd">
    <allow-access-from domain="*" to-ports="*" secure="false"/>
    <site-control permitted-cross-domain-policies="all"/>
    <allow-http-request-headers-from domain="*" headers="*"/>
</cross-domain-policy>

Final Step – Enable the remote storage profile
1. Click on the dropdown box next to your new storage profile in the Remote Storage Profiles page in Kaltura Admin Console

2. Select “Export Automatically” and then click “OK”

3. You will receive the confirmation that your storage was autoed 🙂

Test your new configuration

You can go ahead and test your new configuration. Upload a new video in the KMC, let it convert, and wait for it to get distributed. After that, try to play the entry and analyze it in your favorite sniffer. You should see that the movies are being downloaded from your cloudfront CDN, look for flv and mp4 files.

Good Luck

Vadim Tarasov

Software developer, PandaOS

8 thoughts on “Setting Up Kaltura S3 Cloudfront CDN – CE 9 External Storage With CDN Delivery

  1. Very helpful, saved me a lot of time.

    Just wanted to point out that #7 under “Setting up the Remote Storage Profile in the Admin Console” is not available as an option when I am setting up a remote storage profile. There is nowhere in the modal window for entering an HTTP Delivery Base URL. I am running v9.18.

    1. Hey Nic,
      Sorry for the late answer.
      HTTP Delivery Base URL is a required field under ‘Delivery Details’ section. Did you manage to find it?

      Vadim.

  2. Hi thanks for this, it’s exactly what I need to do: send videos from a bucked through CloudFront so they are accesible for a user in a browser.

    A couple of questions though:
    1) The guide seems to jump to the Remote Storage Profile section, and I’m not actually sure what this is.
    2) It also doesn’t mention the point where the AMI’s are set up with PandaOS and how the above relates to that. Are there some pre-requisits from the earlier post that are required?

    Many Thanks!

    1. Hey Rich,
      1. Remote storage profile is a Kaltura feature which allows you to export your Kaltura content to a different server via multiple options: FTP, SFTP, Amazon S3 and more. This post is about setting up your remote storage and Cloudfront at Amazon.
      2. No prerequisites required to set up remote storage profile.

  3. Hey again,

    Further to my last comment, I now have an amazon instance running in AWS of (ami-1392aa7a) which is Panda CE 9. I can connect to it via putty but expected to be able to get to the Admin Console, in order to set up the Remote Storage profile, by hitting http://ec2-….compute-1.amazonaws.com/start, as I had read this worked in other threads.

    Alas, the only thing I can see at this link (minus the start) is the Linux Test page. Any ideas?

    Many thanks and great blog post.

    R

  4. Hey man,

    Thanks for your HowTo. It saves lives. But, for now, they changed the interface in Kaltura CE 10. Now they have something called “Delivery Profiles” and I can’t seem to get it right.

Leave a Reply

Your email address will not be published. Required fields are marked *