Deploy a Kaltura CE5 Cluster on AWS

The Kaltura CE5 video server can be deployed as a stand alone server, but to serve many users a distributed setup is required. Setting up a Kaltura CE5 cluster on AWS also enables adding more servers when needed with no impact on availability.

At PandaOS we wrote a web interface for making such deployments automatic and easy with using Amazon Web Services. Below I describe the setup required on AWS to install a distributed Kaltura environment.

Before you start the installation, you need to create an AMI (Amazon image) of a machine with Kaltura installed on it. Then you can use that image to launch all your Kaltura instances.

AWS Kaltura Cluster Structure Overview

We use the VPC (Virtual Private Cloud) service to divide the cluster into a private and public subnets to restrict access to most of the instances. A typical AWS Kaltura cluster includes the following instances:

  1. Main front – includes the API access, sphinx indexing process and database and the data warehouse (analytics) database.
  2. Additional fronts – to distribute load.
  3. Encoding batch – a batch instance for encoding.
  4. Additional batch – for the rest of the tasks.
  5. Master MySQL database – for Kaltura use.
  6. Slave MySQL database – for replication.
  7. NFS – an instance with attached shared storage accessed by the front and batch instances.
  8. ElastiCache instance – for memcache.
  9. NAT Instance – Connects the public and private subnets.
  10. Template instance – cloned to create new instance when needed, and for SSHing into the other machines.
In addition, you can use S3 for the video storage combined with the Cloudfront CDN.
A diagram of the architecture is shown below:

The setup includes several steps:

  1. Setting up a Virtual Private Cloud. This enables us to isolate some of our servers and resources to increase security.
  2. Creating a Key Pair.
  3. Configuring the NAT instance. The NAT instance enables outbound access to all instances.
  4. Launching a template instance. This machine would be used for maintenance access and for deploying the other servers.
  5. Creating a Load Balancer.
  6. Preparing the template instance for deployment of other instances.
  7. Launching a memcache instance.

1. Setting up a Virtual Private Cloud

  • We start by logging into the AWS console.
  • Click on VPC.
  • Select a region from the top right dropdown. Make sure that the desired region is selected through the process.
  • Create a VPC with public and private subnets. The instances in the private subnet can access the internet through a NAT (Network Address Translation).
  • Select Security Groups and click on the Inbound tab.
  • Create a new rule for All Traffic, click Add Rule and Apply Rule Changes (Please note that the source for the rule must be 0.0.0.0/0).

2. Creating a Key Pair

  •  Select the EC2 service and click on Key Pairs.
  • Create and download a key pair. This key pair will be used in all the instances.

3. Configuring the NAT instance

  •  Select the EC2 service and click on Instances.
  • Label the NAT instance, for example “NAT – DO NOT DELETE”.
  • Right click NAT instance. Select change termination protection and in the dialog Yes, enable.

4. Launching a template instance

  • Select the EC2 service and enter AMIs.
  • Launch the image with your kaltura installed.
  • Set instance type to Template M1 Small
  • Set Launch into to EC2-VPC subnet 10.0.0.0.
  • Click continue on the following screens. When prompted for name, enter “Kaltura Template – DO NOT DELETE”.
  • Create a new key pair and download it. This key pair will be used for all instances.
  • Choose default security group.
  • Launch the instance.
  • Go to VPC and click on Elastic IPs.
  • Create an elastic ip.
  • Associate the second elastic ip with the kalura template instance. Write down this ip address.

5. Creating a Load Balancer

  • Go to EC2Load Balancers and create a load balancer.
  • Name it “kaltura-lb”.
  • Choose Create LB Inside vpc.
  • Set the ping path to “/start/index.php”.
  • Add subnet 10.0.0.0 to selected subnets.
  • Create new security group named “kaltura-lb”.

6. Preparing the template instance for deployment of other instances

  • Open a terminal.
  • Copy the key downloaded previously to ~/.ssh.
  • Change the key file permissions: chmod 600 [key file]
  • To log in to the machine, use ssh [key file] ubuntu@[template instance ip]

7. Launching a memcache instance (optional)

  • Go to the ElastiCache service.
  • Launch a cache cluster with one node. A cache.t1.micro instance should be enough.
  • Edit your Kaltura front server’s kConf file and add the configuration endpoint.

Now you can launch your kaltura instances in the private subnet (10.0.0.0) from the template machine!

3 thoughts on “Deploy a Kaltura CE5 Cluster on AWS

  1. Pingback: Introduction to the Panda Kaltura AWS Cluster Panda OS Consulting and Development

    1. Hi Kinx,
      Sadly we don’t have a public AMI we can share at the moment, feel free to contact us and I can explain through Skype.
      We should release such AMI in the near future.

      I’ll try to post here but we will post about it here so come again :).

Leave a Reply

Your email address will not be published. Required fields are marked *