First Time Use#

If you are the person who requested the Trel Instance, you are the administrator. Follow the instructions below.

Starting Trel platform (Adminstrator only)#

The Trel platform would have sent you an e-mail with trel_rsa.pem. If not, you would be able to download it from the trelcloud.com portal.

For In Linux/OSX,

> dos2unix trel_rsa.pem
> chmod 400 trel_rsa.pem

When this Trel instance is initially delivered to you, it lacks credentials, so most processes are in SHUTDOWN mode.

The following credentials will be required before we can start the processes.

AWS credentials (access_key, secret_key). Trel uses AWS S3 for multiple internal steps as well as logs. It takes your credentials to use your buckets for storing these.
Credentials for SMTP access. (email, password, server)
OpenAI API key for the AI powered data discovery.

To configure the instance and start all processes login and run setup command as follows. Find the Trel IP in the e-mail you redeived or trelcloud.com.

> ssh -i trel_rsa.pem trel_admin@<trel ip>

> first_time_trel_setup

This command will ask you for the required credentials and finally end with this message:

Your Trel instance is fully configured and started up.

Please continue to the instructions for installing the CLI.

At this point, run the following command and keep the output handy for subsequent steps.

trel show_key

Using treladmin, administrators of multi-user plans can now,

Add more groups (see group_add subcommand)
Assign permissions to groups (see permission and group_update subcommand)
Add users (see user_add subcommand)
Assign users to groups (see user_update subcommand)
Enable authentication for users (see user_update subcommand)

Welcome e-mail for users#

Your Trel administrator will add you as a user using your e-mail ID. They will permit email based authentication for a limited time window of say, 48 hours.

At this point, you will get an e-mail with

tunnel_only_rsa
Link to SDK + CLI wheel package

Rename the tunnel_only_rsa file to trel_rsa.pem and follow instructions below.

Administrators will also receive the email. They should note the link, but can otherwise skip this step.

CLI Installation & Setup#

Interacting with Trel as a user requires CLI installation and UI access. Let us take a look at installing the CLI and creating access keys for users.

Step 1: Virtual Environment

Setup a virtual environment and activate it

Ensure Python3 is installed. 3.9 or higher is recommended.

> python3 -m venv trelcli
> cd trelcli\Scripts
> activate.bat

> python3 -m venv trelcli
> . trelcli/bin/activate

Step 2: Install CLI

Download the wheel file from the email by clicking the link. Install the wheel package using the following command:

(trelcli) > pip install <wheel file>

This will give you access to,

trel command

ds_sequence command. This produces date sequences that simplify custom input selection for Trel CLI.

Step 3: Set up the tunnel

A number of interfaces are hosted on the Trel instance. However, for security reasons, only port 22 (SSH) is open on this instance. To access the interfaces, you must set up a tunnel to the Trel instance.

The port forwardings can be left as default if those ports are open. If those ports are currently taken, you can pick any other port.

(trelcli) > trel tunnel trel_rsa
No user config found. Creating one now.
Please enter the domain/IP-address of the Trel instance: <IP from Email>
Please enter the port [5440]: 8000
Please enter the workspace port [8001]:
Please enter the docs port [8002]:
Please enter the trel logs port [8003]:
Please enter the trel sensor port [8004]:
Please enter the trel worker port [8005]:
Config created.

If everything is successful, this command will hang and keep the port open. Your settings have been saved and next time you run this command no questions will be asked.

Step 4: Authentication

To trigger authentication, try to run a valid trel command:

(trelcli) > trel datasets
No login credentials found in user config.
Please enter an e-mail address: <your registered email>

The next step depends on whether or not you have already generated keys. If you are the administrator, you have already generated the keys.

For this you must be within the authentication window.

Confirmation code has been sent to your e-mail.
Please enter it here: <6 digit code>
Login successful. Credentials saved to config file

This will create a key and store it in a configuration file. A user will have only one key. To login from other machines, you need to provide this key. To see it, run this command

(trelcli) > trel show_key
-----BEGIN PUBLIC KEY-----
*******
*******
*******
*******
-----END PUBLIC KEY-----
(trelcli) >

This can happen outside the authentication window.

It looks like you already have generated the keys.
It should be re-used. If you don't have it, ask the admin to re-enable authentication for you.
Please enter it here (press enter to skip).
public_key: <paste key generated from "First time new key" process>
Login successful. Credentials saved to config file.

Now, all trel commands will be accessible.

(trelcli) > trel jobs
┌────────┬───────────┬──────────────────────┬──────────┐
│ name   │ is_active │ is_scheduler_enabled │ favorite │
├────────┼───────────┼──────────────────────┼──────────┤
│ my_job │         1 │                    1 │          │
└────────┴───────────┴──────────────────────┴──────────┘

Logging in to Web-UI#

Once you have the tunnel and CLI working, you have access to the UI.

Verify the tunnel command is running.
Run trel show_key to show the public key created. Copy it.
Go to http://localhost:5440/settings.html (change the port if you changed it for the tunnel)
Paste the public key into the appropriate text-box. For “Trel User ID”, enter your email.
Click “Save and go back”. Go to http://localhost:5440/, and you should not get any authentication errors.

If you made a mistake in the key or the user_id, going to the home page will show encryption errors. Redo the steps correctly to resolve this.

Trel Interfaces#

If you configured the ports as suggested, the following interfaces would be available:

Interface	Description
http://localhost:5440/	Trel Web-UI
http://localhost:8001/	Provides access to the folders created to execute the attempts. Access attempts logs and sensor logs here as well.
http://localhost:8002/	Documentation specific to your release of Trel
http://localhost:8003/	Access the Trel system log. Covers the REST API and scheduling system.
http://localhost:8004/	Access the Trel sensor log. Logs from the sensor sub-system. If your sensor can’t start, find the reason here.
http://localhost:8005/	Access the Trel worker log. Includes logs from workers that run jobs. If your job can’t start, perhaps because the file name is wrong, you will find the error here.

Initial Status of Trel#

Assuming you chose the basic plan, and AWS cloud, using the information you provided, the Trel platform has been configured with two repositories: dw which is based on Athena and dl which is based on S3.

You have access to various sensors that can crawl or extract-load from data sources and add to catalog. these sensors must be created by filling in the details in the template after clicking the “Add Sensors” button in the sensor page.

You have access to four execution technologies: python, athena, emr_pyspark, and ec2. The first one runs in the Trel instance, while the rest run in the cloud. There are multiple templates for each and you can create new jobs using these technologies by filling in the templates. Start by clicking “Add job” in the jobs page.

Troubleshooting and Support#

If you run into any issues, go to our discord channel Trel Support. You can point out bugs, suggest improvements, get clarifications regarding best practices, and get architectural advice. Let us know how you feel about Trel and the catalog-based dependency mechanism.