How to run a Python script

Back to Tutorials
Svg background
Create and run your first Python script
To create a Python script, in the automation menu, click on "New automation", then choose "Python script".
Create a python script
Then, the python editor is displayed where you can modify the default python script (i.e. a "Hello world"):
Default python script
To run this "Hello world" example, open the "Execution" tab and click on Run Now. The script is put in the execution queue and as soon as it is finished, its log will be displayed at the top of the "Last 10 executions" table:
Default python script
Use the Serenytics Python client to create an email alert

The most classical use case for Python scripts is to run a script manipulating data stored in Serenytics.

To do this, the easiest way is to use our Python client to access datasources and query their data. It also provides helper functions such as sending an email, refreshing dashboard cache, getting data as pandas dataframe... You can also directly use our REST API, but using our client makes it a lot easier. The full documentation of our Python client is here.

Here is a very simple example to get the number of rows in a datasource and send an email if it is above a threshold:

import serenytics
client = serenytics.Client()

ds = client.get_data_source_by_uuid(uuid="c5d540df-38dc-45be-abf6-b7969add39ff")

nb_rows = ds.get_nb_rows()
if(nb_rows > 5_000_000):
    subject="[Alert] too many rows"
    recipients=["adrien.auclair@serenytics.com"]
    html="Nb of rows: {0}".format(nb_rows)
    client.send_email(subject=subject, recipients=recipients, html=html)

A more useful example is to query the data with filters to get the number of rows in the current day:

import serenytics
client = serenytics.Client()

ds = client.get_data_source_by_uuid(uuid="c5d540df-38dc-45be-abf6-b7969add39ff")

options = {
            'order': 'row_by_row',
            'data_processing_pipeline': [{
                'select': [{'name': '$$count$$', 'agg_function': 'sum'}],
                'where': [{'name': 'Date', 'op': 'in', 'value': 'day(0)'}],
                'limit': 100,
            }]
        }

data = ds.get_data(options=options)
if data.rows:
    nb_alerts_today = data.rows[0][0]
else:
    nb_alerts_today = 0

if(nb_alerts_today > 10):
    subject="Alerte, too many alerts today"
    recipients=["adrien.auclair@serenytics.com"]
    html="Nb of alerts: {0}".format(nb_alerts_today)
    client.send_email(subject=subject, recipients=recipients, html=html)
Use the Serenytics client to get data as Pandas dataframes
Pandas is a classical library to manipulate data in Python. With Serenytics, you can query any of your datasources to get data as a Pandas dataframe; then manipulate this dataframe and upload the result in a Serenytics storage.
import serenytics
client = serenytics.Client()

# input datasource
ds = client.get_data_source_by_uuid(uuid="c5d540df-38dc-45be-abf6-b7969add39ff")
df = ds.get_data().get_as_dataframe()

# manipulate your dataframe in Pandas

# upload the resulting data in a Serenytics storage
storage = client.get_data_source_by_uuid(uuid="e4695cbb-5779-4402-9a12-7d4b9e2d32d3")
storage.reload_data_from_dataframe(df)
Run the scripts locally

If you need to write advanced Serenytics Python scripts, a good strategy is to tune them locally before running them on our platform.

To run a script locally, you need to install the Serenytics package on your machine and to provide your API key when you create the serenytics Client object in your script (see here for details).

Running the script on your machine lets you use a debugger (we use PyCharm at Serenytics) to add breakpoints and execute your script step by step. You can also use git (or any other versioning system) to keep track of the evolution of your scripts.

The underlying technology

When you run a Python script in Serenytics, it is executed within a dedicated Docker container. The Python environment in this container includes the most used packages for data processing and many day-to-day packages:

  • numpy
  • pandas
  • scipy
  • scikit-learn
  • xlrd
  • sqlalchemy
  • psycopg2
  • pymysql
  • pymongo
  • requests
  • paramiko
  • boto3
  • cchardet
  • oauth2client
  • python-dateutil
  • monthdelta
  • gevent
  • jinja2
  • premailer
  • robobrowser
  • currencyconverter

The memory allowed for your execution is limited. And there is also a timeout to limit the duration of the execution. You need to contact us if you have specific package needs or need to increase the memory available or the timeout of your scripts.

×