pyreports

pyreports

pyreports is a python library that allows you to create complex report from various sources

What is pyreports?

pyreports wants to be a library that simplifies the collection of data from multiple sources such as databases, files and directory servers (through LDAP), the processing of them through built-in and customized functions, and the saving in various formats (or, by inserting the data in a database).

Designed for a data scientist

Simplifying the life of those who collect and process data

Time saver

It saves time, without worrying about creating connections or managing files manually.

PEP 8

Each row of code is PEP 8 compliant

Features

Get Started

Installation

Install pyreports is easy with pip.


    $ pip install pyreports
                     

Basic usage

I take the data from a database table, filter the data I need and save it in a csv file.

    
import pyreports

# Select source: this is a DatabaseManager object
mydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')

# Get data
mydb.execute('SELECT * FROM site_login')
site_login = mydb.fetchall()

# Filter data
error_login = pyreports.Executor(site_login)
error_login.filter([400, 401, 403, 404, 500])

# Save report: this is a FileManager object
output = pyreports.manager('csv', '/home/report/error_login.csv')
output.write(error_login.get_data())
    

Advanced usage

In this example we will analyze and capture parts of a web server log. For each error code present in the log, we will create a report that will be inserted in a book, where each sheet will contain the details of the error code. In the last sheet, there will be an element counter for every single error present in the report.


import pyreports
import tablib

# Get apache log data: this is a FileManager object
apache_log = pyreports.manager('log', '/var/log/httpd/error.log')
# Read log based on regexp
data_log = apache_log.read('([(\d\.)]+) - - \[(.*?)\] "(.*?)" (\d+) - "(.*?)" "(.*?)"',
                        headers=['ip', 'date', 'operation', 'url', 'code', 'client'])

# Create a collection of Report objects
all_apache_error = pyreports.ReportBook(title='Apache error on my site')

# Create a Report object based on error code
all_error = set(data_log['code'])
for code in all_error:
    all_apache_error.add(pyreports.Report(data_log, filters=[code], title=f'Error {code}'))

# Count all error code
counter = pyreports.counter(data_log, 'code')
# Append new Report on ReportBook with error code counters
error_counter = tablib.Dataset(counter.values(), headers=counter)
all_apache_error.add(pyreports.Report(error_counter))

# Save ReportBook on Excel
all_apache_error.export('/home/report/apache_log_error_code.xlsx')
                    

Full Documentation

The complete documentation is much more comprehensive and can be found by clicking the button below.

More on ReadTheDocs

License

pyreports is free and open source.

The license is GPLv3 and you can consult it directly here: Github License

Donate: PayPal

If you are interested in improvements or have innovation proposals, or simply want to help improve the library, you can fork the project from here! Thank you for your support!

Contact

I hope you find this python library useful.
Feel free to get in touch if you have any questions or suggestions.

Matteo Guadrini aka GU

Love pyreports?

If you are interested in other projects, follow me on Github or Linkedin. You can also email me if you like!

Matteo Guadrini
DevOps and Python developer

Get Connected