Setting up Swank integration for HLS streams

Introduction: 

The client receives a physical disk with movies from Swank. These movies need to be copied to our IPTV server to be accessible and playable by users through our IPTV system. The disk contains a predefined structure of files and folders, allowing our integration script to ingest them efficiently. The objective of the script is to automatically ingest movies into the server, eliminating the need for manual insertion of movies and their metadata by our clients. This manual is intended to assist the IT team in the process of ingesting movies into the server.

Step-by-step instructions:

Here are the step by stem instructions how to prepare the disk and how to successfully ingest the content from the swank movies. The manual is written how to ingest the swank movies HLS type, specifically the format for video is .mpd.

1. First setup environment on the server for Swank input:

Objective:

Prepare the scripts and folders on the server in advance to avoid losing time when the new disk with the new content arrives for ingestion. Ensure that the scripts and folders are set up in the correct server and in the appropriate locations. 

Scripts you need to set up only at the begining of the project. On exisitng project where this is already set, you can skip that point and go to next one on mounting the disk.

Precondition: 

You must have root access to the server.

You need to have a terminal of the server open.

Steps: 

Create a folder where you will temporarily mount the connected disk.

Command:

Upload our custom swank script to the root folder on the server where the content will be. If the MW and VOD servers are separate, run the script on the VOD server.

On the NAS in the folder \RR\Integracije\Swank\HLS, there is a zip file named Swank-HLS-importer.zip. Copy the file to the server and unzip it with the MC tool and place it in the /root/ folder, so the path will be /root/swank_importer/swank.sh.

Result: 

You have new folder /temp-swank created.

You have a script on the server in path /root/swank_importer/swank.sh

2. Connecting and mounting the Disk

Objective:

The new disk that requires copying to the system must be physically connected to the appropriate server and then correctly mounted to the designated folder.

Prerequisite:

The server must be correctly identified.

The server must have an available port to insert the disk.

Steps: 

Make sure the customer connects the disk to the VOD server using a port located at the back. If available, opt for the blue port.

Once the customer confirms the disk is connected, you can verify on the server if it has been detected.

Command to verify detected devices:

In the image provided below, you can observe that the new disk has been detected as sdc1, and the size matches the connected disk.

Open

Now we will mount the disk to the /temp-swank folder on the server with the command:

Result:

After completing the process, the Mount point column should display the path /temp-swank, confirming the successful connection between the disk and the folder. By navigating to the temp-swank folder on the server, you can access the content stored on the disk.

3. Copying Content

Objective: 

The entire folder needs to be copied to the server into the /data/content/swank/ new folder you created.

Precondition: 

The disk needs to be mounted to the server.

You need access to the server so you can execute the commands.

Steps:

First, go to the server in the /data/content/ and create a swank folder and inside swank folder a new folder with today's date. The name of the folder inside is not important, but we decided it will be the date of the content upload. 

For new installations you can create the folders, for existing projects it is better to check first where the existing movies are. They could be somewhere else. For Crystal Cruises, they are in /data/swank. For majority they are in /data/content/swank.

You can connect to the server with WinSCP and check and create the folders, or with an SSH terminal.

Example:

Go to the content folder and check if it lists the folders.

Commands:

Create a new folder and check if it is displayed:

Our folder name is /data/swank/20240716/ and the goal would be to copy new files inside it.

Using the MC (Midnight Commander) tool, simply copy the content from the disk to the server.

Start the screen tool, which means we will run commands in the screen tool, allowing us to close our screen while the command (process) still runs in the background. We do this because copying the movies will take about 3 hours, and we don't want to keep the screen open all the time. Also, the connection might break in between, and we don't want to repeat the process.

Commands:

Then, in the blue screen, we get two halves. In one half, go to the /temp-swank folder, and in the other half, go to /data/swank/20240716 folder.

Move between the folders using the TAB key. Then, go to the folder you want to copy and select all files inside it using the Insert key (press the key multiple times to select all files, they will turn yellow). Then press the F5 key to start copying. In the example below, the content is already copied, and since this is the Crystal project, the folder path is a bit different.

Open

Result:

When you see the identical content on your final destination the folder you created in data/content/swank/ it means the task is successfully done.

4 Content metadata insertion into Axon manager

Objective:

Insert the movie metadata into the system with the script, precisely to Axon manager. Metadata includes the title of the movie, description, genres, images, and link to the video file.

Precondition:

The movies are copied in the folder on the server.

Steps required:

Open the script /root/swank_importer/swank.sh and update the link to the new folder.

Since this example is a Crystal project, the path to the Swank folder is /data/swank/. Otherwise, the default path is /data/content/swank/

This project uses HLS and links to the movie using extensions exo+, https, and a domain instead of an IP address. Here we define how the link will be entered into the manager, and the last part of the line is the path to the files we want to ingest.

In the example below, for the Crystal project, we only need to change the last folder from 20231209 to 20240716 in both places.

Open

Save the script and open a terminal to run it.

Commands:



The process will start and finish in a few minutes. You will see it adding images and metadata. If there is an error, the script will stop. Image below shows how it successfully finishes.

Open

Result: 

Go to the manager and refresh the movie table. The new movies should appear but will not yet be added to categories (Group column), so you can easily identify the new ones. If they do not appear, it indicates that the script did not finish or there was an error.

5 Categorizing content

Objective:

The script inserted the metadata into the manager, but it did not map it to the current category table in the IPTV system. This step involves running a script to map the new movies to the existing categories. You can always edit the script to add new categories if needed.

Precondition: 

Movies are already inserted in the manager and seen. Only missing is the categories set to them. We do see the genres of the movies in the Genre column, but the Group column is empty.

Open

Steps required: 

1. Connect to the MW server and open the SSH terminal. 

2. Write te command to run the script:

Check if it did categorize the movies. You can always add new Video groups in the IPTV admin and run the comand again.

6 Disconnecting the Disk

Objective: 

After the disk is copied and the content is uploaded, it is essential to properly unmount the disk before disconnecting it. Disconnecting the disk without unmounting it first can cause issues when connecting a new disk, and may require a server reboot to fix.

Precondition: 

Disk is still mounted.

We don't need the disk to be mounted any more.

Steps required::

Use the command to check the disks and see the name it is using:

Command:

Use the command to unmount the disk, replacing "sdc1" with your disk's name:

Command:

The command should complete on its own. If it says the task is busy, you need to close open windows or use a forceful unmount:

Command:

Result: 

You can run the command lsblk again and you will see there is no mount point folder written next to the disk. The same way as it was on the beginning before the disk was mounted.

Additional information:

Detail description and explanation of the commands  in the swank.sh file 

All possible parameters to set in the  *swank_to_axon.py script are:

• --add-exo = If this parameter is in the line you set it means it will add the exo+ prefix in front of the link in movie and trailer external link column. 

• --mp4 = If this is present, the script things it is a mp4 fle and not HLS. If no mp4, then it considers it is a HLS.

• --content-domain CONTENT_DOMAIN = With this parameter we can set the domain of the link. In our nginx the share folder is linked to the /data/ folder so this link needs to contain a path to the folder inside the /data/ folder. Means the path from data folder to the end file you want. Example:  https://content.nevronitv.si/content/swank/20231208/) is pointing to the folder that is inside the data folder. The whole URL is like this: https://content.nevronit.si/content/swank/20231208/Publicity/123Trailer.mp4 and the device goes to this folder to take the file:  /data/content/swank/20231208/Publicity/123Trailer.mp4.

• --dry-run = If this parameter is present, then script only shows in terminal what is is parsing. It doesn't actually ingest the data in the manager. It only shows in the terminal what it would inset. It would insert the images, title, description. With this parameter we  just check if the parser works and if the link we created is ok. Otherwise if you had an errro in the link, you would need to delete those movies from the manager, edit the script and run it again.

• Path to XML = The last parameter at the end is a mandatory one. You need to write the path to the folder where the Manifest.xml file is located. This way the script knows where the file is and ingest the metadata. The script will check in all subfolders for all the XML files.

FAQ:

Question: How to upload a Swank movie?

Answer:

Swank movies are uploaded using a script for quicker insertion. Please refer to the manual on Swank integration for detailed instructions.

Question: How to use the Screen tool?

Answer:

To use the Screen tool, follow these steps:

1. Type the command screen in the terminal to enter screen mode.

2. Execute the commands you want.

3. To exit the screen, press CTRL + A then D. This will detach the screen session.

4. To reattach to the screen session, type screen -R. If you have multiple screen sessions active, it will ask you to specify the screen session number. Example: screen -R 938484.

5. To stop the screen session because you no longer need it, type exit.

For more information or help, you can always search online, as Screen is a commonly used tool on Linux servers.

Question: How to use MC (Midnight Commander) to copy files on a Linux server?

Answer:

1. Open Midnight Commander:

• Type `mc` in the terminal and press Enter to open Midnight Commander.

2. Navigate to Source Directory:

• Use the arrow keys (or mouse) to navigate to the directory containing the files you want to copy. 

• Press Enter to open folders.

3. Select Files to Copy:

• Navigate to each file you want to copy and press the Insert key to select it. Selected files will be highlighted.

4. Navigate to Destination Directory:

• Press the TAB key to switch to the other panel (other side).

• Use the arrow keys to navigate to the destination directory where you want to copy the files.

5. Copy Files:

• Once you are in the destination directory, press the TAB key to switch back to the source panel.

• Press `F5` to open the copy dialog.

• Confirm the copy action by pressing Enter.

6. Exit Midnight Commander:

• Press `F10` to exit Midnight Commander.

For more detailed help, you can always refer to the online documentation or use the `F1` key within Midnight Commander for context-sensitive help.

Question: How to check if a disk is detected on the server?

Answer: 

To check if a disk is detected on a Linux server:

1. Use the `lsblk` Command: Open a terminal and run `lsblk`. This command lists all block devices, including disks, and their details like size and mount points.

2. Check System Logs: Use `dmesg` command to view kernel ring buffer messages. Look for disk detection messages or errors that might indicate if the disk is recognized.

3. Check `/dev/` Directory: Navigate to the `/dev/` directory and look for entries like `sda`, `sdb`, etc., which represent disks and partitions. Their presence indicates detection.

These steps should help you determine if the disk is detected and available for use on the Linux server.

Question: How to check if a movie link is correct?

Answer:  

To verify if a movie link is correct:

1. Connect to any server in the IPTV network.

2. Copy the entire movie link from the manager, including "exo+" prefix if present.

3. Remove any prefix (like "exo+") from the link.

4. Open a terminal and use the `wget` command followed by the cleaned-up link.

Example:

wget https://content.nevronit.si/content/swank/20231208/Publicity/123Trailer.mp4

If the terminal displays "loaded 100%" after running the command, the file is reachable and has successfully downloaded to the current location. If the link is incorrect or the file is not found, an error message will indicate the issue.

Question: How to check free space on the Linux server?

Answer:

To check available disk space on a Linux server:

1. Open the terminal.

2. Enter the command `df -h` and press Enter. This command displays disk usage statistics in a human-readable format.

3. Look for the `/data` mountpoint to determine how much free space is available. This is where the movies are stored.

This will show you the amount of free space specifically available for storing movies on your server.

Troubleshooting articles:

We don't detect the disk the client said they connected.

The copying of the disk failed and showing error Not enough space?

Related Articles:

Importing Swank movies format mp4

Importing Anuvu movies