Step 1:Load data

Introduction

Loading data is an important first step. Without data, you cannot take the next step. In the DataOwn BI platform, you can load data without writing complicated code, just by using a few clicks. In practice, you often need coordination with the administrator of a data source before creating a new connection.

Creating a data source

You do this by selecting "+ Add data source" (top right) in the "Data sources" table. Then you can choose from connections with databases or with APIs. Within databases you have a choice between various types of SQL (PostgreSQL, MySQL, MariaDB and MS SQL Server).

• The "display name" and "connector description" are only visible within DataOwn and help maintain an overview.
• The administrator of the source should know or generate the other information. In many cases, this administrator should also whitelist IP address 212.114.98.137 (for the development system: 82.25.101.130) so the Meltano instance of DataOwn can make a connection.

After this, it is recommended to test the connection with "test connection" before choosing "Save data source". If you do this, you can then select the tables and the columns within them. If you don't select anything, only the primary key will be included, so always make a selection. Within a table you can select all columns via "select all" or "deselect all". You can return to this screen by clicking on "Open catalog".

We recommend loading columns with data that is traceable to a person only if they are necessary. You can load them later if needed, but it is always better not to load columns such as personal names.

We have not yet been able to test other connectors, such as API connectors, often due to a lack of a test system. Let us know if you have a specific system that we can use for testing and/or to create a standard setup.

The standard settings for a new data source (contoso on a PostgreSQL database) work and lead to a small test dataset.

Actually loading data

You set the schedule and start a manual run directly on the data source itself. For the schedule you can choose "Manual", "Daily", or "Weekly". With weekly, you choose which day the job runs. Daily and weekly loads run at night, somewhere between 0:00 and 4:00. From the data source you can also start a manual run right away — also when a daily or weekly schedule is already set, if you need an extra run.

In the Monitor you can see the status of all runs, and you can also kick off a new run there.